Nova Resource:Cvn/SAL

<mediawiki xmlns="http://www.mediawiki.org/xml/export-0.11/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mediawiki.org/xml/export-0.11/ http://www.mediawiki.org/xml/export-0.11.xsd" version="0.11" xml:lang="en">
  <siteinfo>
    <sitename>Wikitech</sitename>
    <dbname>labswiki</dbname>
    <base>https://wikitech.wikimedia.org/wiki/Main_Page</base>
    <generator>MediaWiki 1.46.0-wmf.26</generator>
    <case>first-letter</case>
    <namespaces>
      <namespace key="-2" case="first-letter">Media</namespace>
      <namespace key="-1" case="first-letter">Special</namespace>
      <namespace key="0" case="first-letter" />
      <namespace key="1" case="first-letter">Talk</namespace>
      <namespace key="2" case="first-letter">User</namespace>
      <namespace key="3" case="first-letter">User talk</namespace>
      <namespace key="4" case="first-letter">Wikitech</namespace>
      <namespace key="5" case="first-letter">Wikitech talk</namespace>
      <namespace key="6" case="first-letter">File</namespace>
      <namespace key="7" case="first-letter">File talk</namespace>
      <namespace key="8" case="first-letter">MediaWiki</namespace>
      <namespace key="9" case="first-letter">MediaWiki talk</namespace>
      <namespace key="10" case="first-letter">Template</namespace>
      <namespace key="11" case="first-letter">Template talk</namespace>
      <namespace key="12" case="first-letter">Help</namespace>
      <namespace key="13" case="first-letter">Help talk</namespace>
      <namespace key="14" case="first-letter">Category</namespace>
      <namespace key="15" case="first-letter">Category talk</namespace>
      <namespace key="110" case="first-letter">Obsolete</namespace>
      <namespace key="111" case="first-letter">Obsolete talk</namespace>
      <namespace key="112" case="first-letter">OfficeIT</namespace>
      <namespace key="113" case="first-letter">OfficeIT talk</namespace>
      <namespace key="116" case="first-letter">Tool</namespace>
      <namespace key="117" case="first-letter">Tool talk</namespace>
      <namespace key="498" case="first-letter">Nova Resource</namespace>
      <namespace key="499" case="first-letter">Nova Resource Talk</namespace>
      <namespace key="666" case="first-letter">Heira</namespace>
      <namespace key="667" case="first-letter">Heira Talk</namespace>
      <namespace key="710" case="first-letter">TimedText</namespace>
      <namespace key="711" case="first-letter">TimedText talk</namespace>
      <namespace key="828" case="first-letter">Module</namespace>
      <namespace key="829" case="first-letter">Module talk</namespace>
    </namespaces>
  </siteinfo>
  <page>
    <title>Nova Resource:Cvn/SAL</title>
    <ns>498</ns>
    <id>1516</id>
    <revision>
      <id>2408819</id>
      <parentid>2406178</parentid>
      <timestamp>2026-05-03T13:39:48Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>Krinkle: Disable &quot;Admin immed notify&quot; for cvn-private https://lists.wikimedia.org/postorius/lists/cvn-private.lists.wikimedia.org/settings/automatic_responses. We previously removed the sub form but this is no longer supported in mailman3. We require confirm/moderate for new subs, there is no way to turn it off. But we can at least disable the noise.</comment>
      <origin>2408819</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="25357" sha1="9mnoybig6xfpr8ugta44fe6o7pmfenw" xml:space="preserve">=== 2026-05-03 ===
* 13:39 Krinkle: Disable &quot;Admin immed notify&quot; for cvn-private https://lists.wikimedia.org/postorius/lists/cvn-private.lists.wikimedia.org/settings/automatic_responses. We previously removed the sub form but this is no longer supported in mailman3. We require confirm/moderate for new subs, there is no way to turn it off. But we can at least disable the noise.

=== 2026-04-27 ===
* 12:22 Krinkle: /cs flags #cvn-meta NathanVeritas voiced

=== 2026-04-01 ===
* 13:34 AntiComposite: restart all bots

=== 2026-02-04 ===
* 20:33 AntiComposite: Restart all bots

=== 2025-12-26 ===
* 15:54 Operator873: /cs flags #cvn-zh-scan nya_1F616EMO voiced

=== 2025-11-27 ===
* 13:48 AntiComposite: CVNBot10 load tok.wikipedia tok: ([[phab:T404567|T404567]])
* 13:47 AntiComposite: CVNBot9 load ms.wikiquote q:ms: ([[phab:T404700|T404700]])
* 13:45 AntiComposite: CVNBot8 load min.wikisource s:min: ([[phab:T408343|T408343]])
* 13:44 AntiComposite: CVNBot7 load pcm.wikiquote q:pcm: ([[phab:T408351|T408351]])
* 13:43 AntiComposite: CVNBot6 load tl.wikisource s:tl: ([[phab:T388654|T388654]])
* 13:42 AntiComposite: CVNBot10 load bew.wiktionary wikt:bew: ([[phab:T402134|T402134]])
* 13:41 AntiComposite: CVNBot9 load zgh.wiktionary wikt:zgh: ([[phab:T399785|T399785]])
* 13:40 AntiComposite: CVNBot8 load min.wikibooks b:min: ([[phab:T395499|T395499]])
* 13:38 AntiComposite: CVNBot7 load rki.wikipedia rki: ([[phab:T392499|T392499]])
* 13:37 AntiComposite: CVNBot6 load mad.wikisource s:mad: ([[phab:T391767|T391767]])

=== 2025-10-28 ===
* 23:16 AntiComposite: /cs flags #cvn-commons revi local_op

=== 2025-08-20 ===
* 20:35 AntiComposite: CVNBot10 load nup.wikipedia nup: ([[phab:T390711|T390711]])

=== 2025-07-11 ===
* 14:38 AntiComposite: cvn-app10 restart all bots
* 11:10 AntiComposite: cvn-app12 restart all bots
* 11:09 AntiComposite: cvn-app10 restart all bots

=== 2025-06-20 ===
* 20:49 AntiComposite: cvn-app12: restart all bots
* 20:48 AntiComposite: cvn-app10: restart all bots

=== 2025-05-26 ===
* 17:59 Krinkle: Create cvn-app14 (debian-12.0-bookworm, g4.cores2.ram4.disk20)
* 17:59 Krinkle: Create cvn-app13 (debian-12.0-bookworm, g4.cores2.ram4.disk20)
* 17:57 Krinkle: Delete cvn-apache10 instance (replaced/shutdown 2 days ago), ref [[phab:T395164|T395164]]

=== 2025-05-23 ===
* 20:30 Krinkle: Shut off cvn-apache10, [[phab:T395164|T395164]]
* 20:29 Krinkle: Change cvn.wmcloud.org web proxy from cvn-apache10 to cvn-apache11,  [[phab:T395164|T395164]]
* 20:22 Krinkle: Change cvn.wmflabs.org web proxy from cvn-apache10 to cvn-apache11,  [[phab:T395164|T395164]]
* 19:45 Krinkle: Create cvn-apache11 (debian-12.0-bookworm, g4.cores2.ram4.disk20), [[phab:T395164|T395164]])

=== 2025-05-16 ===
* 18:22 Krinkle: Replace outreach.wikipedia with outreach.wikimedia in cvn-sw/CVNBot19 per https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/820245 since the source channel was renamed
* 17:30 Krinkle: krinkle@cvn-apache10:/srv/cvn/git/infrastructure$ git pull -- Deploy https://gerrit.wikimedia.org/r/1146724
* 17:30 Krinkle: krinkle@cvn-apache10 Update git remote in /srv/cvn/git/infrastructure from github.com/countervandalism to https://gerrit.wikimedia.org/r/labs/countervandalism/cvn-infrastructure

=== 2025-04-21 ===
* 17:22 AntiComposite: Hard reboot cvn-app10, flapping and not responsive to ssh

=== 2025-03-30 ===
* 06:55 Krinkle: krinkle@cvn-apache10: Run `sudo chmod 644 /srv/cvn/git/infrastructure/crontab-config/*.cron`, per [[phab:T390415|T390415]]

=== 2025-03-12 ===
* 02:18 AntiComposite: CVNBot9 load id.wikivoyage voy:id: ([[phab:T381080|T381080]])
* 02:15 AntiComposite: CVNBot8 load tig.wikipedia tig: ([[phab:T381379|T381379]])
* 02:14 AntiComposite: CVNBot7 load knc.wikipedia knc: ([[phab:T385185|T385185]])
* 02:11 AntiComposite: CVNBot6 load syl.wikipedia syl: ([[phab:T386464|T386464]])
* 02:08 AntiComposite: CVNBot10 load sat.wiktionary wikt:sat: ([[phab:T386631|T386631]])

=== 2025-02-03 ===
* 22:05 AntiComposite: Hard reboot cvn-apache10 from Horizon per https://lists.wikimedia.org/hyperkitty/list/cloud-announce@lists.wikimedia.org/message/ZOCPVXX6BKLC76OHMIQW26YLBCKEBTGQ/
* 21:58 AntiComposite: Hard reboot cvn-app10 from Horizon per https://lists.wikimedia.org/hyperkitty/list/cloud-announce@lists.wikimedia.org/message/ZOCPVXX6BKLC76OHMIQW26YLBCKEBTGQ/

=== 2025-01-02 ===
* 12:46 Krinkle: /cs flags #cvn-wp-en Lordseriouspig voiced
* 12:45 Krinkle: /cs flags #cvn-sw Lordseriouspig voiced

=== 2024-11-23 ===
* 00:41 AntiComposite: CVNBot9 load ka.wikisource s:ka: ([[phab:T363243|T363243]])
* 00:38 AntiComposite: CVNBot8 load tcy.wikisource s:tcy: ([[phab:T378471|T378471]])
* 00:37 AntiComposite: CVNBot7 load tcy.wiktionary wikt:tcy: ([[phab:T378463|T378463]])
* 00:25 AntiComposite: Upgrade CVNBot29 to v4.0.4
* 00:25 AntiComposite: Upgrade CVNBot28 to v4.0.4
* 00:24 AntiComposite: Upgrade CVNBot27 to v4.0.4
* 00:24 AntiComposite: Upgrade CVNBot26 to v4.0.4
* 00:23 AntiComposite: Upgrade CVNBot25 to v4.0.4
* 00:23 AntiComposite: Upgrade CVNBot24 to v4.0.4
* 00:22 AntiComposite: Upgrade CVNBot23 to v4.0.4
* 00:22 AntiComposite: Upgrade CVNBot22 to v4.0.4
* 00:21 AntiComposite: Upgrade CVNBot19 to v4.0.4
* 00:21 AntiComposite: Upgrade CVNBot17 to v4.0.4
* 00:21 AntiComposite: Upgrade CVNBot16 to v4.0.4
* 00:20 AntiComposite: Upgrade CVNBot10 to v4.0.4
* 00:19 AntiComposite: Upgrade CVNBot9 to v4.0.4
* 00:19 AntiComposite: Upgrade CVNBot8 to v4.0.4
* 00:19 AntiComposite: Upgrade CVNBot7 to v4.0.4
* 00:17 AntiComposite: Upgrade CVNBot6 to v4.0.4

=== 2024-11-22 ===
* 23:52 AntiComposite: Upgrade CVNBot21 to v4.0.4
* 23:51 AntiComposite: Upgrade CVNBot20 to v4.0.4
* 23:51 AntiComposite: Upgrade CVNBot18 to v4.0.4
* 23:51 AntiComposite: Upgrade CVNBot15 to v4.0.4
* 23:50 AntiComposite: Upgrade CVNBot14 to v4.0.4
* 23:50 AntiComposite: Upgrade CVNBot13 to v4.0.4
* 23:49 AntiComposite: Upgrade CVNBot12 to v4.0.4
* 23:48 AntiComposite: Upgrade CVNBot11 to v4.0.4
* 23:47 AntiComposite: Upgrade CVNBot5 to v4.0.4
* 23:45 AntiComposite: Upgrade CVNBot3 to v4.0.4
* 23:44 AntiComposite: Upgrade CVNBot2 to v4.0.4
* 23:41 AntiComposite: Upgrade CVNBot1 to v4.0.4
* 23:32 AntiComposite: Upgrade CVNBot4 to v4.0.4
* 17:08 AntiComposite: restart CVNBots on cvn-app12 due to simultaneous RCReader failure 91950.519949 seconds

=== 2024-11-08 ===
* 23:24 AntiComposite: Restarting all CVNBots due to simultaneous RCReader disconnect 54323.128318 seconds ago

=== 2024-10-29 ===
* 20:56 AntiComposite: add sh.wikipedia to CVNBot6 as #cvn-wp-sh didn't survive the libera migration
* 14:22 AntiComposite: restart all CVNBots

=== 2024-10-28 ===
* 12:50 AntiComposite: restarting all CVNBots, not coming up cleanly

=== 2024-10-25 ===
* 02:23 AntiComposite: add cs.wikivoyage to CVNBot10 ([[phab:T370913|T370913]])
* 02:21 AntiComposite: add bdr.wikipedia to CVNBot9 ([[phab:T371760|T371760]])
* 02:18 AntiComposite: add mos.wikipedia to CVNBot8 ([[phab:T374644|T374644]])
* 02:14 AntiComposite: add kge.wikipedia to CVNBot7 ([[phab:T374815|T374815]])
* 02:11 AntiComposite: add rsk.wikipedia to CVNBot6 ([[phab:T375017|T375017]])
* 02:07 AntiComposite: add mad.wiktionary to CVNBot9 ([[phab:T375024|T375024]])
* 02:06 AntiComposite: add gor.wikiquote to CVNBot8 ([[phab:T375095|T375095]])
* 02:04 AntiComposite: add nr.wikipedia to CVNBot7 ([[phab:T375102|T375102]])
* 02:01 AntiComposite: add tdd.wikipedia to CVNBot6 ([[phab:T375424|T375424]])
* 01:54 AntiComposite: add shn.wikinews to CVNBot9 ([[phab:T375433|T375433]])
* 01:52 AntiComposite: add iba.wikipedia to CVNBot8 ([[phab:T376572|T376572]])
* 01:50 AntiComposite: add bcl.wikisource to CVNBot7 ([[phab:T377088|T377088]])
* 01:47 AntiComposite: add ann.wikipedia to CVNBot6 ([[phab:T377160|T377160]])
* 01:43 AntiComposite: add igl.wikipedia to CVNBot9 ( [[phab:T363263|T363263]] )
* 01:41 AntiComposite: add my.wikisource to CVNBot8 ([[phab:T363270|T363270]])
* 01:39 AntiComposite: add foundation.wikimedia to CVNBot19
* 01:38 AntiComposite: add wikitech.wikimedia to CVNBot19

=== 2024-10-24 ===
* 11:36 AntiComposite: restart all CVNBots

=== 2024-10-23 ===
* 17:33 AntiComposite: restart all CVNBots

=== 2024-07-03 ===
* 02:00 AntiComposite: add kus.wikipedia to CVNBot7 ([[phab:T360303|T360303]])
* 01:57 AntiComposite: add bew.wikipedia to CVNBot6 ([[phab:T360310|T360310]])
* 01:54 AntiComposite: add ms.wikisource to CVNBot9 ([[phab:T363250|T363250]])
* 01:53 AntiComposite: add kaa.wiktionary to CVNBot8 ([[phab:T363256|T363256]])
* 01:50 AntiComposite: add dtp.wikipedia to CVNBot7 ([[phab:T365230|T365230]])
* 01:48 AntiComposite: add btm.wikipedia to CVNBot6 ([[phab:T368067|T368067]])
* 01:45 AntiComposite: add fon.wikipedia to CVNBot9 ([[phab:T347939|T347939]])
* 01:43 AntiComposite: add blk.wikisource to CVNBot8 ([[phab:T343542|T343542]])
* 01:41 AntiComposite: su.wikisource to CVNBot7 ([[phab:T343548|T343548]])
* 01:39 AntiComposite: add tly.wikipedia to CVNBot6 ([[phab:T345170|T345170]])
* 01:37 AntiComposite: add dga.wikipedia to CVNBot9 ([[phab:T350229|T350229]])
* 01:35 AntiComposite: add bjn.wikiquote to CVNBot8 ([[phab:T350235|T350235]])
* 01:32 AntiComposite: add zgh.wikipedia to CVNBot7 ([[phab:T350241|T350241]])
* 01:28 AntiComposite: add bbc.wikipedia to CVNBot6 ([[phab:T350373|T350373]])

=== 2024-06-24 ===
* 16:40 Krinkle: cvn-clerkbot parts #cvn-unifications (not operated by CVN, renamed to #wikimedia-unifications)

=== 2024-06-18 ===
* 08:52 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_project_to_ovs (exit_code=0)
* 08:48 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_project_to_ovs

=== 2024-03-22 ===
* 05:30 Operator873: /cs flags #cvn-simplewikis Drummingman +voice

=== 2024-02-28 ===
* 21:34 Krinkle: /cs flags #cvn-wp-da Sarrus local_op

=== 2024-01-11 ===
* 12:19 AntiComposite: /cs flags #cvn-meta Bsadowski1 local_op

=== 2023-12-01 ===
* 15:30 AntiComposite: restart everything after WMCS network outage

=== 2023-10-07 ===
* 14:50 AntiComposite: kill 2 CVNBot11 processes and restart, bot not joined to IRC

=== 2023-09-22 ===
* 00:06 Op873: /cs flags #cvn-wp-en Oshwah +AV

=== 2023-09-16 ===
* 10:33 JackSparrow: /cs flags #cvn-wp-fa Arian_Ar local_op

=== 2023-09-07 ===
* 01:35 AntiComposite: restart all cvn-app12 bots
* 01:33 AntiComposite: restart all cvn-app10 bots

=== 2023-08-15 ===
* 14:44 AntiComposite: reboot cvn-app10 from Horizon, bots dead and not responding to SSH

=== 2023-08-09 ===
* 00:07 AntiComposite: add 9 wikis to #cvn-sw (ref [[phab:T332379|T332379]] [[phab:T336115|T336115]] [[phab:T332093|T332093]] [[phab:T332093|T332093]] [[phab:T335987|T335987]] [[phab:T334459|T334459]] [[phab:T333271|T333271]] [[phab:T334740|T334740]] [[phab:T342865|T342865]])

=== 2023-08-08 ===
* 23:46 AntiComposite: drop wo.wikiquote from CVNBot10 (closed) [[phab:T334482|T334482]]

=== 2023-07-27 ===
* 18:15 AntiComposite: Kill and restart CVNBot29 on cvn-app12

=== 2023-07-06 ===
* 16:21 AntiComposite: point git repos to gerrit on cvn-app10
* 16:19 AntiComposite: point git repos to gerrit on cvn-app12
* 16:03 AntiComposite: CVNBot v4.0.3 deployed to all bots ([[phab:T327126|T327126]], [[phab:T327127|T327127]])
* 16:01 AntiComposite: Upgrade CVNBot29 to v4.0.3
* 16:00 AntiComposite: Upgrade CVNBot28 to v4.0.3
* 16:00 AntiComposite: Upgrade CVNBot27 to v4.0.3
* 15:59 AntiComposite: Upgrade CVNBot26 to v4.0.3
* 15:58 AntiComposite: Upgrade CVNBot25 to v4.0.3
* 15:57 AntiComposite: Upgrade CVNBot24 to v4.0.3
* 15:57 AntiComposite: Upgrade CVNBot23 to v4.0.3
* 15:55 AntiComposite: Upgrade CVNBot22 to v4.0.3
* 15:54 AntiComposite: Upgrade CVNBot19 to v4.0.3
* 15:53 AntiComposite: Upgrade CVNBot17 to v4.0.3
* 15:46 AntiComposite: Upgrade CVNBot16 to v4.0.3
* 15:44 AntiComposite: Upgrade CVNBot10 to v4.0.3
* 15:41 AntiComposite: Upgrade CVNBot9 to v4.0.3
* 15:40 AntiComposite: Upgrade CVNBot8 to v4.0.3
* 15:39 AntiComposite: Upgrade CVNBot7 to v4.0.3
* 15:38 AntiComposite: Upgrade CVNBot6 to v4.0.3
* 04:37 AntiComposite: Upgrade CVNBot21 to v4.0.3
* 04:34 AntiComposite: Upgrade CVNBot20 to v4.0.3
* 04:33 AntiComposite: Upgrade CVNBot18 to v4.0.3
* 04:30 AntiComposite: Upgrade CVNBot15 to v4.0.3
* 04:23 AntiComposite: Upgrade CVNBot14 to v4.0.3
* 04:22 AntiComposite: Upgrade CVNBot13 to v4.0.3
* 04:14 AntiComposite: Upgrade CVNBot12 to v4.0.3
* 04:09 AntiComposite: Upgrade CVNBot11 to v4.0.3
* 04:03 AntiComposite: Upgrade CVNBot5 to v4.0.3
* 04:01 AntiComposite: Upgrade CVNBot4 to v4.0.3
* 04:00 AntiComposite: Upgrade CVNBot3 to v4.0.3
* 03:57 AntiComposite: Upgrade CVNBot2 to v4.0.3
* 03:51 AntiComposite: Upgrade CVNBot1 to v4.0.3

=== 2023-06-28 ===
* 02:34 Operator873: /cs flags #cvn-sw Fehufanga voiced

=== 2023-06-16 ===
* 22:05 AntiComposite: manually restart cvn-clerkbot

=== 2023-05-15 ===
* 14:58 hauskater: Dropped akwiki and nawiki from CVNBot10 as closed wikis. On-wiki lists require an update.

=== 2023-04-26 ===
* 20:07 AntiComposite: /cs flags #cvn-mk-scan M4r51n voiced

=== 2023-04-21 ===
* 22:12 Operator873: granted voice to Fehufanga in #cvn-simplewikis

=== 2023-04-14 ===
* 18:28 AntiComposite: restart cvn-app10 from horizon, bots quit and ssh times out

=== 2023-03-22 ===
* 03:33 Operator873: Voiced Tulsi in #cvn-sw -meta -mediawiki -commons -simplewikis

=== 2023-03-13 ===
* 19:46 Operator873: CVNBot18 restarted

=== 2023-03-03 ===
* 14:45 AntiComposite: /cs flags #cvn-sw-spam COIBot bot

=== 2023-02-27 ===
* 22:33 herzog: Loaded gur.wikipedia to SWMT Group 4 (CVNBot9) - [[phab:T327842|T327842]]
* 18:04 herzog: Loaded guc.wikipedia to CVNBot9 / Group 4 - [[phab:T326236|T326236]]

=== 2023-02-02 ===
* 00:21 ma: Added 12 new wikis to CVNBot&lt;nowiki&gt;{&lt;/nowiki&gt;6,7,8&lt;nowiki&gt;}&lt;/nowiki&gt;, 4 to each one. Refs.: [[phab:T321283|T321283]] [[phab:T321289|T321289]] [[phab:T321295|T321295]] [[phab:T326139|T326139]] [[phab:T305281|T305281]] [[phab:T310873|T310873]] [[phab:T312215|T312215]] [[phab:T314640|T314640]] [[phab:T314646|T314646]] [[phab:T316457|T316457]] [[phab:T317113|T317113]] [[phab:T319191|T319191]]

=== 2023-01-30 ===
* 22:50 Krinkle: Delete cvn-app8 and cvn-app9 instances, ref [[phab:T306066|T306066]]

=== 2023-01-28 ===
* 02:51 AntiComposite: /cs flags #cvn-sw Ajraddatz local_op

=== 2023-01-24 ===
* 08:54 Krinkle: Delete cvn-apache9, [[phab:T306066|T306066]]
* 08:54 Krinkle: Suspend cvn-app8 and cvn-app9 (`pgrep -af cvn` is empty on both), [[phab:T306066|T306066]]

=== 2023-01-23 ===
* 16:53 AntiComposite: Deploy {{Gerrit|716e140}} to app12 ([[phab:T306066|T306066]])
* 16:50 AntiComposite: Deploy {{Gerrit|716e140}} to app9 ([[phab:T306066|T306066]])
* 16:29 AntiComposite: Deploy {{Gerrit|442f324}} to app12 ([[phab:T306066|T306066]])
* 16:25 AntiComposite: Deploy {{Gerrit|442f324}} to app9 ([[phab:T306066|T306066]])
* 16:01 AntiComposite: Deploy {{Gerrit|9024b8f}} to app12 ([[phab:T306066|T306066]])
* 15:59 AntiComposite: Deploy {{Gerrit|9024b8f}} to app9 ([[phab:T306066|T306066]])

=== 2023-01-22 ===
* 21:40 AntiComposite: start cvndb-CVNBot14-publish on app10
* 21:07 AntiComposite: Deploy {{Gerrit|1acdb8e}} to cvn-app10, starting bots ([[phab:T306066|T306066]])
* 20:56 AntiComposite: disable cvndb-CVNBot14-publish on app8
* 20:51 AntiComposite: Deploy {{Gerrit|1acdb8e}} to cvn-app8, stopping bots ([[phab:T306066|T306066]])
* 19:53 AntiComposite: Deploy {{Gerrit|80ea1f5}} to cvn-app10 ([[phab:T306066|T306066]])
* 15:43 AntiComposite: restart all CVNBots on app9
* 15:42 AntiComposite: restart all CVNBots on app8

=== 2023-01-17 ===
* 00:15 Krinkle: Suspend cvn-apache9, replaced by cvn-apache10, ref [[phab:T306066|T306066]]
* 00:14 Krinkle: Switch cvn.wmflabs.org from cvn-apache9 to cvn-apache10

=== 2023-01-16 ===
* 00:10 Krinkle: Move https://github.com/countervandalism/cvn-clerkbot to https://github.com/wikimedia/countervandalism-cvn-clerkbot (with HTTP and Git redirect preserved), and replace with Gerrit mirror

=== 2023-01-15 ===
* 23:12 Krinkle: Create 'labs-cvn' permission group in Gerrit with CVN staff members
* 23:12 Krinkle: Move https://github.com/countervandalism/cvn-api to https://github.com/wikimedia/countervandalism-cvn-api (with HTTP and Git redirect preserved), and replace with Gerrit mirror
* 22:02 Krinkle: Switch new cvn.wmcloud.org proxy from cvn-apache9 to cvn-apache10 (Leave main cvn.wmflabs.org as-is for now).

=== 2023-01-14 ===
* 21:45 AntiComposite: move cvn-clerkbot from cvn-app9 to cvn-app12 (deploy {{Gerrit|4cee27a}})
* 21:22 AntiComposite: move cvn-clerbot back to cvn-app9 (deploy {{Gerrit|371ba2a}})
* 21:10 AntiComposite: move cvn-clerkbot from cvn-app9 to cvn-app12 (deploy {{Gerrit|3f3f40f}})

=== 2023-01-10 ===
* 23:22 Krinkle: krinkle@cvn-apache9$ update infrastructure.git, sudo apachectl graceful
* 23:20 Krinkle: Create cvn.wmcloud.org web proxy (in addition to cvn.wmflabs.org)

=== 2023-01-07 ===
* 20:53 AntiComposite: apply role::labs::lvm::srv only to cvn-apache9, cvn-app8, and cvn-app9 to fix puppet failures on new instances

=== 2023-01-04 ===
* 20:47 Krinkle: Allocate new floating IPs to cvn-app10 and cvn-app11
* 20:46 Krinkle: Create new cvn-apache10, cvn-app10, cvn-app11 with Debian 11 Bullseye to replace the old Debian 9.1 Stretch instances
* 20:04 taavi: bump floating ip quota from 2 to 4, [[phab:T326269|T326269]]

=== 2022-12-27 ===
* 20:11 Frosty873: /cs flags #cvn-meta xaosflux voiced
* 20:11 Frosty873: /cs flags #cvn-wp-en xaosflux voiced

=== 2022-12-23 ===
* 03:25 AntiComposite: /cs flags #cvn-meta tryvix1509 voiced
* 03:24 AntiComposite: /cs flags #cvn-mediawiki tryvix1509 voiced
* 03:24 AntiComposite: /cs flags #cvn-sw tryvix1509 voiced

=== 2022-10-18 ===
* 23:13 Joan: CVNBot3 restarted (Last message was received on RCReader 62854.814658 seconds ag)

=== 2022-09-04 ===
* 22:21 Operator873: /cs flags #cvn-simplewikis Enfcer +AV
* 02:20 Operator873: /cs flags #cvn-sw Bot873 +voiced

=== 2022-08-26 ===
* 14:09 hauskatze: Loaded pcm.wikipedia and guw.wiktionary to CVNBot8 &amp; 9 respectively {{!}} [[phab:T310880|T310880]] [[phab:T309057|T309057]]

=== 2022-07-09 ===
* 16:42 AntiComposite: /cs flags #cvn-commons pandakekok9 voiced

=== 2022-07-08 ===
* 21:53 Krinkle: krinkle@horizon.wikimedia.org Add anticomposite  as project member and project admin to cloudvps.cvn

=== 2022-07-01 ===
* 21:39 Krinkle: cvn-app8: kill CVNBot14.exe and two (!) procs for CVNBot18.exe

=== 2022-06-25 ===
* 03:25 AntiComposite: /cs flags #cvn-wp-en PhantomTech voiced

=== 2022-06-22 ===
* 21:04 op873: &lt;+CVNBot3&gt; Added: LuchoCR is on es.wikipedia bot list, added by Operator873{{!}}CVN until the end of time (&quot;Mass blockiing P2P-proxies with script&quot;)
* 20:34 op873: restart CVNBot3 (possibly caused by block flood)
* 19:31 op873: restart CVNBot3

=== 2022-06-15 ===
* 18:49 AntiComposite: /cs flags #cvn-wp-en Zppix voiced
* 18:48 AntiComposite: /cs flags #cvn-simplewikis Zppix voiced

=== 2022-05-23 ===
* 00:24 Joan: Flags +AV were set on Sargento in cvn-wp-es
* 00:23 Joan: Flags +AV were set on alhen in cvn-wp-es

=== 2022-05-19 ===
* 23:10 Joan: CVNBot3 restarted (Last message was received on RCReader 92593.747667 seconds ago)

=== 2022-05-11 ===
* 07:34 Operator873: /cs flags #cvn-wp-en Tamzin voiced

=== 2022-05-07 ===
* 17:40 Operator873: /cs flags #cvn-sw koi voiced
* 17:39 Operator873: /cs flags #cvn-zh-scan koi voiced

=== 2022-04-28 ===
* 03:19 Joan: CVNBot3 restarted (Last message was received on RCReader 75273.332577 seconds ago)

=== 2022-04-22 ===
* 15:08 AntiComposite: /cs flags #cvn-meta Bsadowski1 voiced

=== 2022-04-18 ===
* 20:44 AntiComposite: /cs flags #cvn-sw Vermont voiced

=== 2022-04-13 ===
* 22:40 Operator873: /cs flags #cvn-meta Joan voiced
* 22:40 Operator873: /cs flags #cvn-sw Joan voiced
* 22:14 Joan: CVNBot3 restarted (Last message was received on RCReader 54942.175428 seconds ago)

=== 2022-04-07 ===
* 23:15 Operator873: /cs flags #cvn-wp-hr NovakWatchmen local_op
* 23:13 Operator873: voiced Superpes (Superpes15) in #cvn-sw #cvn-sw-spam and #cvn-it-scan

=== 2022-04-04 ===
* 17:34 Operator873: Voiced Vermont in #cvn-meta and #cvn-simplewikis /cs flags #cvn-meta Vermont voiced

=== 2022-03-30 ===
* 14:33 Joan: CVNBot3 restarted (Last message was received on RCReader 26318.335196 seconds ago)

=== 2022-03-28 ===
* 02:38 AntiComposite: /cs flags #cvn-wp-en Bsoyka voiced

=== 2022-03-21 ===
* 20:22 Operator873: /cs flags #cvn-simplewikis Bsadowski1 +AfiotvV
* 20:17 Operator873: Operator873{{!}}CVN (Operator873) set flags +AVfitv on Bsadowski1
* 20:03 Operator873: Operator873{{!}}CVN (Operator873) set flags +V on Bsadowski1
* 17:04 AntiComposite: /cs flags #cvn-sw Bsadowski1 local_op

=== 2022-03-15 ===
* 15:38 Joan: CVNBot3 restarted (Last message was received on RCReader 26424.279343 seconds ago)

=== 2022-03-14 ===
* 14:02 Joan: CVNBot3 restarted (Last message was received on RCReader 17096.72183 seconds ago)

=== 2022-03-12 ===
* 16:27 Joan: CVNBot3 restarted (Last message was received on RCReader 27236.775673 seconds ago)

=== 2022-03-11 ===
* 14:24 Joan: CVNBot3 restarted (Last message was received on RCReader 18853.006849 seconds ago)

=== 2022-03-10 ===
* 14:08 Joan: CVNBot3 restarted (Last message was received on RCReader 22518.614282 seconds ago)

=== 2022-03-08 ===
* 20:27 AntiComposite: /cs flags #cvn-wp-en Sarrus voiced
* 20:27 AntiComposite: /cs flags #cvn-simplewikis Sarrus voiced
* 20:27 AntiComposite: /cs flags #cvn-commons Sarrus voiced

=== 2022-03-07 ===
* 16:30 AntiComposite: /cs flags #cvn-meta zabe voiced
* 16:25 AntiComposite: /cs flags #cvn-simplewikis DannyS712 voiced
* 16:25 AntiComposite: /cs flags #cvn-meta DannyS712 voiced
* 16:25 AntiComposite: /cs flags #cvn-sw TheresNoTime voiced
* 16:07 Krinkle: /cs flags #cvn-staff Operator873 staff
* 16:07 Krinkle: /cs flags #cvn-staff AntiComposite staff

=== 2022-03-05 ===
* 04:13 Joan: CVNBot3 restarted (Last message was received on RCReader 31573.894101 seconds ago)

=== 2022-03-03 ===
* 16:39 Joan: CVNBot3 restarted (Last message was received on RCReader 36578.236383 seconds ago)

=== 2022-03-01 ===
* 13:21 Joan: CVNBot3 restarted (Last message was received on RCReader 20646.781861 seconds ago)

=== 2022-02-15 ===
* 14:12 Joan: CVNBot3 restarted (Last message was received on RCReader 25001.391103 seconds ago)

=== 2022-02-13 ===
* 18:47 andrewbogott: switching to project-local nfs server cvn-nfs-1
* 17:54 andrewbogott: switching to project-local nfs server puppet-diffs-nfs-1

=== 2022-02-10 ===
* 16:17 Joan: CVNBot3 restarted (Last message was received on RCReader 39817.871151 seconds ago)

=== 2022-02-08 ===
* 15:51 Joan: CVNBot3 restarted (Last message was received on RCReader 28868.916144 seconds ago)

=== 2022-02-04 ===
* 23:59 andrewbogott: accidentally restarted all VMs due to misreading the project purge page.  sorry!

=== 2022-02-02 ===
* CVN: Several bots restarted after netsplit took nickserv and some bots with it.
* 10:26 Krinkle: CVNBot1 bes del delete(?!d) —  originally added by huh (reason: &quot;widewuto&quot;)

=== 2022-02-01 ===
* 15:20 Joan: CVNBot3 restarted (Last message was received on RCReader 26990.323435 seconds ago)

=== 2022-01-31 ===
* 17:37 Joan: CVNBot3 restarted (Last message was received on RCReader 48827.882566 seconds ago)

=== 2022-01-27 ===
* 16:58 Joan: CVNBot3 restarted (Last message was received on RCReader 29206.852828 seconds ago)

=== 2022-01-21 ===
* 16:07 Joan: CVNBot3 restarted (Last message was received on RCReader 22091.557102 seconds ago)

=== 2022-01-20 ===
* 18:13 Cam11598: CVNBot15 restarted

=== 2022-01-19 ===
* 17:26 Joan: Restarted CVNBot3 (Last message was received on RCReader 28129.031916 seconds ago)

=== 2022-01-18 ===
* 16:55 Joan: Restarted CVNBot3 (Last message was received on RCReader 26283.381782 seconds ago)

=== 2022-01-17 ===
* 16:33 Joan: Restarted CVNBot3 (#cvn-wp-es) (Last message was received on RCReader 197065.877109 seconds ago)

=== 2022-01-15 ===
* 04:56 Cam11598: restarted CVNBOT18 8:55:47 PM &lt;�25B100+ CVNBot18&gt; Last message was received on RCReader 29723.456263 seconds ago

=== 2022-01-13 ===
* 01:29 Cam11598: restarted CVNBot2 nickserv issue
* 01:29 Cam11598: restarted CVNBot18 - no response from RC feed

=== 2022-01-09 ===
* 18:18 Joan: Flags +AV were set on Hasley in cvn-wp-es (sysop at es.wikipedia)
* 17:56 Krinkle: /cs flags #cvn-wp-es Joan local_op

=== 2022-01-07 ===
* 22:08 hauskatze: CVNBot9 load co.wiktionary wikt:co:
* 22:04 hauskatze: CVNBot9 load ban.wikisource s:ban:
* 22:04 hauskatze: CVNBot9 load ba.wikibooks b:ba:
* 10:51 hauskatze: Loaded alt.wikipedia to Group 4 (CVNBot9) - small wiki not monitored

=== 2022-01-06 ===
* 19:42 hauskatze: Loaded ami.wikipedia to CVNBot8 - [[phab:T292421|T292421]]
* 19:41 hauskatze: Loaded pwn.wikipedia to CVNBot7 - [[phab:T292419|T292419]]
* 19:39 hauskatze: Loaded lmo.wiktionary to CVNBot6 - [[phab:T292076|T292076]]
* 19:34 hauskatze: Loaded jv.wikisource to CVNBot6 refs. [[phab:T287319|T287319]]
* 19:29 Krinkle: cs flags #cvn-sw hauskatze local_op
* 13:57 Krinkle: Krinkle added $a:Cam11598 to the #cvn-staff I list (+I)

{{SAL|Project Name=cvn}}
&lt;noinclude&gt;
==Archives==
* [[Nova Resource:Cvn/SAL/Archive 1|Archive 1]] (2006-2009)
* [[Nova Resource:Cvn/SAL/Archive 2|Archive 2]] (2010-2011)
* [[Nova Resource:Cvn/SAL/Archive 3|Archive 3]] (2012-2013)
* [[Nova Resource:Cvn/SAL/Archive 4|Archive 4]] (2013-2021) (some parts in 2013 are not indexed)
[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>9mnoybig6xfpr8ugta44fe6o7pmfenw</sha1>
    </revision>
  </page>
  <page>
    <title>Deployments</title>
    <ns>0</ns>
    <id>4108</id>
    <revision>
      <id>2408874</id>
      <parentid>2408778</parentid>
      <timestamp>2026-05-04T07:35:04Z</timestamp>
      <contributor>
        <username>ScheduleDeploymentBot</username>
        <id>37566</id>
      </contributor>
      <comment>Add [[gerrit:1281965]] to Monday, May 04 UTC afternoon backport window</comment>
      <origin>2408874</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="43301" sha1="7wr3wxfyj6lru6nhvz4ubkr1lh9087h" xml:space="preserve">{{Navigation MediaWiki deployment}}
This page tracks '''upcoming''' '''deployments''' of software to the [[:m:Special:SiteMatrix|Wikimedia Foundation servers]].

== Getting started ==
Ensure you joined the {{irc|wikimedia-operations}} IRC channel as all deployment-related communications happen there.
If you need help, contact [[:mw:Wikimedia Release Engineering Team|Release Engineering]] on IRC at {{irc|wikimedia-releng}}; and ping Tyler (&lt;code&gt;thcipriani&lt;/code&gt;).

* '''MediaWiki is deployed weekly''' through the [[/Train|Deployment Train]]. Other services follow their own schedule.
* '''Times are pinned to San Francisco''', thus the UTC time changes in March and November per [[:en:Daylight saving time in the United States|DST]].
* '''Prefer regular [[Backport windows]]''' over adding new windows. To request deployment of a config change or backport, add your username and Gerrit URL to one of the backport windows on this page. You must be online in #wikimedia-operations on IRC during your deployment and install [[WikimediaDebug]] ahead of time. The #wikimedia-operations channel requires you to [[:m:IRC/Instructions#Register your nickname, identify, and enforce|register your nickname]] before you can join.
** You can use the '''backport scheduling tool''' to more easily edit this page: &lt;div style=&quot;text-align: center; margin: 1em 0&quot;&gt;{{Clickable button 2|:toollabs:schedule-deployment|Schedule a backport|class=mw-ui-progressive}}&lt;/div&gt;
* Tasks that meet [[/Inclusion criteria|Inclusion criteria]] '''require their own windows''', which includes long-running tasks. '''Schedule more time''' than you think you need to account for delays and set backs, we recommend one hour for most tasks.
**To create or modify a recurring deploy window, send a patchset to [[:gitlab:repos/releng/release/-/blob/main/make-deployment-calendar/deployments-calendar.yaml|deployments-calendar.yaml file]] in &lt;code&gt;repos/releng/release.git&lt;/code&gt;.
**To create an one-off window, simply edit this page accordingly
** '''Announce''' changes to the [[mail:ops|ops mailing list]] ahead of time if you anticipate or are uncertain about noticeable impacts to database load, HTTP caching, or the introduction of new cookies.
** '''Announce''' deployments of major features to the community via [[:m:Tech/News/Next|Tech News]] and/or via other [[:mw:Wikimedia_Product_Guidance/Communication_channels|Product communication channels]].
* '''Something went wrong?''' See [[Incident response]]. Is there a user-impacting problem? Communicate in the {{irc|wikimedia-operations}} IRC channel. If there is a Phabricator task, ensure [[:phab:tag/wikimedia-incident/|#Wikimedia-Incident]] is tagged, and consider setting the [[:mw:Phabricator/Project_management#Priority_levels|Unbreak Now]] priority.
__TOC__
{{anchor|Next Week|Near Term|Near term|Near-term}}{{clear}}
[[Category:Deployment]]
{{Note|content=Subscribe in Google Calendar via &lt;code&gt;wikimedia.org_rudis09ii2mm5fk4hgdjeh1u64@group.calendar.google.com&lt;/code&gt;.&lt;br&gt;This may not include one-off windows. '''If there are differences, then the wiki page is canonical and correct'''.}}

==Week of May 04==


==={{Deployment_day|date=2026-05-03}}===
{{Deployment calendar event card
    |when=2026-05-03 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}
==={{Deployment_day|date=2026-05-04}}===
{{Deployment calendar event card
    |when=2026-05-04 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|xxb|xxb}}
{{deploy|type=config|gerrit=1279477|title=nlwiki: Modify autoconfirmed requirements for nlwiki|status=}} - {{phabricator|T424898}}
{{ircnick|nya_1F616EMO|1F616EMO}}
{{deploy|type=config|gerrit=1281965|title=zhwikinews: (1/2) revert 20th anniversary logo change (config)|status=}} - {{phabricator|T420165}}
{{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-04 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-04 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|manfredi|manfredi}}
{{deploy|type=1.46.0-wmf.26|gerrit=1281501|title=Email confirmation banner: Remove obsolete arm_b variant|status=}} - {{phabricator|T421366}}
{{deploy|type=1.46.0-wmf.26|gerrit=1281504|title=Use js promise for email confirmation banner|status=}} - {{phabricator|T420007}}
{{ircnick|nya_1F616EMO|1F616EMO}}
{{deploy|type=config|gerrit=1281965|title=zhwikinews: (1/2) revert 20th anniversary logo change (config)|status=}} - {{phabricator|T420165}}
{{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-04 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-04 08:30 SF
    |length=0.5
    |window=Wikimedia Portals Update
    |who={{ircnick|jan_drewniak|Jan Drewniak}}
    |what=Weekly window for the portals page: https://www.wikipedia.org/
}}
{{Deployment calendar event card
    |when=2026-05-04 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-04 10:00 SF
    |length=0.5
    |window=Wikidata Query Service weekly deploy
    |who={{ircnick|ryankemper|Ryan}}
    |what=...
}}
{{Deployment calendar event card
    |when=2026-05-04 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|toyofuku|toyofuku}}
{{deploy|type=config|gerrit=1277667|title=Enable the reading list beta feature survey on all wikipedias|status=}} - {{phabricator|T421776}}
{{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-04 14:00 SF
    |length=2
    |window=Weekly Security deployment window
    |who={{ircnick|Reedy|Sam}}, {{ircnick|sbassett|Scott}}, {{ircnick|Maryum|Maryum}}, {{ircnick|manfredi|Manfredi}}
    |what=Held deployment window for Security-team related deploys.
}}
{{Deployment calendar event card
    |when=2026-05-04 16:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-04 19:00 SF
    |length=1
    |window=Automatic branching of MediaWiki, extensions, skins, and vendor – see [[Heterogeneous deployment/Train deploys]]
    |who=N/A
    |what=Branch &lt;code&gt;wmf/1.47.0-wmf.1&lt;/code&gt;
}}
{{Deployment calendar event card
    |when=2026-05-04 20:00 SF
    |length=1
    |window=Automatic deployment of MediaWiki, extensions, skins, and vendor to testwikis only – see [[Heterogeneous deployment/Train deploys]]
    |who=N/A
    |what=Deploy &lt;code&gt;wmf/1.47.0-wmf.1&lt;/code&gt; to testwikis
}}
{{Deployment calendar event card
    |when=2026-05-04 21:00 SF
    |length=1
    |window=Automatic removal of all obsolete MediaWiki versions from the deployment and bare metal servers (except the most-recent obsolete version)
    |who=N/A
    |what=Runs &lt;code&gt;scap clean auto&lt;/code&gt;
}}
{{Deployment calendar event card
    |when=2026-05-04 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-04 23:00 SF
    |length=0.5
    |window=Primary database switchover
    |who={{ircnick|marostegui|Manuel Arostegui}}, {{ircnick|Amir1|Amir}}, {{ircnick|federico3|Federico Ceratto}}
    |what=Held deployment window for database primary masters maintenance
}}
==={{Deployment_day|date=2026-05-05}}===
{{Deployment calendar event card
    |when=2026-05-05 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-05 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-05 05:00 SF
    |length=1
    |window=Mobileapps/RESTBase/Wikifeeds
    |who=Content Transform Team
    |what=Content transform team node services (mobileapps/wikifeeds)
}}
{{Deployment calendar event card
    |when=2026-05-05 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|MatmaRex|Bartosz}}
{{deploy|type=config|gerrit=1270882|title=Remove temporary `wgOAuth2UsePrefixedSub` feature flag|status=}} - {{phabricator|T417690}}
{{deploy|type=config|gerrit=1271969|title=Move privileged global and local group handling to WikimediaCustomizations|status=}} - {{phabricator|T418507}}
{{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-05 07:00 SF
    |length=0.5
    |window=Test Kitchen UI Deployment Window
    |who=Experimentation Platform Team
    |what=Deployment of Test Kitchen UI (fka MPIC)
}}
{{Deployment calendar event card
    |when=2026-05-05 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-05 08:00 SF
    |length=1
    |window=SRE Collaboration Services office hours
    |who={{ircnick|jelto|Jelto}}, {{ircnick|arnoldokoth|Arnold}}, {{ircnick|mutante|Daniel}}, {{ircnick|arnaudb|Arnaud}}
    |what=Services including Gerrit, Phorge (Phabricator), GitLab
}}
{{Deployment calendar event card
    |when=2026-05-05 09:00 SF
    |length=1
    |window=[[Puppet request window]]&lt;br/&gt;&lt;small&gt;'''(Max 6 patches)'''&lt;/small&gt;
    |who={{ircnick|jhathaway|JHathaway}}, {{ircnick|rzl|Reuven}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to Puppet change''
}}
{{Deployment calendar event card
    |when=2026-05-05 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-05 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-7 Version
    |who={{ircnick|brennen|Brennen}}, {{ircnick|jeena|Jeena}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.46.0-wmf.26-&gt;1.47.0-wmf.1|1.46.0-wmf.26|1.46.0-wmf.26}}
* group0 to [[mw:MediaWiki_1.47/wmf.1|1.47.0-wmf.1]]
* '''Blockers: {{phabricator|T423910}}'''
}}
{{Deployment calendar event card
    |when=2026-05-05 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-05 14:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-05 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
==={{Deployment_day|date=2026-05-06}}===
{{Deployment calendar event card
    |when=2026-05-06 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-06 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-06 04:00 SF
    |length=1
    |window=[[mw:Services|Services]] – [[Citoid]] / [[Zotero]]
    |who=Marielle ({{ircnick|mvolz}})
    |what=See [[mw:Citoid|Citoid]]
}}
{{Deployment calendar event card
    |when=2026-05-06 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-06 07:00 SF
    |length=1
    |window=Wikifunctions Services UTC Afternoon
    |who=Abstract Wikipedia team (Africa, Europe, Eastern Americas)
    |what=Wikifunctions back-end k8s services
}}
{{Deployment calendar event card
    |when=2026-05-06 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-06 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-06 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-7 Version
    |who={{ircnick|brennen|Brennen}}, {{ircnick|jeena|Jeena}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.1|1.46.0-wmf.26-&gt;1.47.0-wmf.1|1.46.0-wmf.26}}
* group1 to [[mw:MediaWiki_1.47/wmf.1|1.47.0-wmf.1]]
* '''Blockers: {{phabricator|T423910}}'''
}}
{{Deployment calendar event card
    |when=2026-05-06 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-06 14:00 SF
    |length=1
    |window=Wikifunctions Services UTC Late
    |who=Abstract Wikipedia team (North and South America)
    |what=Wikifunctions back-end k8s services
}}
{{Deployment calendar event card
    |when=2026-05-06 15:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-06 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-06 23:00 SF
    |length=0.5
    |window=Primary database switchover
    |who={{ircnick|marostegui|Manuel Arostegui}}, {{ircnick|Amir1|Amir}}, {{ircnick|federico3|Federico Ceratto}}
    |what=Held deployment window for database primary masters maintenance
}}
==={{Deployment_day|date=2026-05-07}}===
{{Deployment calendar event card
    |when=2026-05-07 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-07 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-07 05:00 SF
    |length=1
    |window=Mobileapps/RESTBase/Wikifeeds
    |who=Content Transform Team
    |what=Content transform team node services (mobileapps/wikifeeds)
}}
{{Deployment calendar event card
    |when=2026-05-07 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-07 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-07 08:00 SF
    |length=1
    |window=Train log triage
    |who={{ircnick|brennen|Brennen}}, {{ircnick|jeena|Jeena}}
    |what=See [[Heterogeneous deployment/Train deploys#Breakage]]
}}
{{Deployment calendar event card
    |when=2026-05-07 09:00 SF
    |length=1
    |window=[[Puppet request window]]&lt;br/&gt;&lt;small&gt;'''(Max 6 patches)'''&lt;/small&gt;
    |who={{ircnick|jhathaway|JHathaway}}, {{ircnick|rzl|Reuven}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to Puppet change''
}}
{{Deployment calendar event card
    |when=2026-05-07 10:00 SF
    |length=1
    |window=Cloud Services/Technical Documentation weekly deploy (Toolhub, Developer portal, Striker)
    |who={{ircnick|bd808}}
    |what=...
}}
{{Deployment calendar event card
    |when=2026-05-07 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-07 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-7 Version
    |who={{ircnick|brennen|Brennen}}, {{ircnick|jeena|Jeena}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.1|1.47.0-wmf.1|1.46.0-wmf.26-&gt;1.47.0-wmf.1}}
* group2 to [[mw:MediaWiki_1.47/wmf.1|1.47.0-wmf.1]]
* '''Blockers: {{phabricator|T423910}}'''
}}
{{Deployment calendar event card
    |when=2026-05-07 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-07 14:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-07 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
==={{Deployment_day|date=2026-05-08}}===
{{Deployment calendar event card
    |when=2026-05-08 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}
{{Deployment calendar event card
    |when=2026-05-08 04:00 SF
    |length=0.5
    |window=GitLab version upgrades
    |who={{ircnick|jelto|Jelto}}, {{ircnick|arnoldokoth|Arnold}}, {{ircnick|mutante|Daniel}}, {{ircnick|arnaudb|Arnaud}}
    |what=GitLab version upgrades
}}
==={{Deployment_day|date=2026-05-09}}===
{{Deployment calendar event card
    |when=2026-05-09 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}

==Week of May 11==


==={{Deployment_day|date=2026-05-10}}===
{{Deployment calendar event card
    |when=2026-05-10 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}
==={{Deployment_day|date=2026-05-11}}===
{{Deployment calendar event card
    |when=2026-05-11 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-11 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-11 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-11 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-11 08:30 SF
    |length=0.5
    |window=Wikimedia Portals Update
    |who={{ircnick|jan_drewniak|Jan Drewniak}}
    |what=Weekly window for the portals page: https://www.wikipedia.org/
}}
{{Deployment calendar event card
    |when=2026-05-11 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-11 10:00 SF
    |length=0.5
    |window=Wikidata Query Service weekly deploy
    |who={{ircnick|ryankemper|Ryan}}
    |what=...
}}
{{Deployment calendar event card
    |when=2026-05-11 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-11 14:00 SF
    |length=2
    |window=Weekly Security deployment window
    |who={{ircnick|Reedy|Sam}}, {{ircnick|sbassett|Scott}}, {{ircnick|Maryum|Maryum}}, {{ircnick|manfredi|Manfredi}}
    |what=Held deployment window for Security-team related deploys.
}}
{{Deployment calendar event card
    |when=2026-05-11 16:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-11 19:00 SF
    |length=1
    |window=Automatic branching of MediaWiki, extensions, skins, and vendor – see [[Heterogeneous deployment/Train deploys]]
    |who=N/A
    |what=Branch &lt;code&gt;wmf/1.47.0-wmf.2&lt;/code&gt;
}}
{{Deployment calendar event card
    |when=2026-05-11 20:00 SF
    |length=1
    |window=Automatic deployment of MediaWiki, extensions, skins, and vendor to testwikis only – see [[Heterogeneous deployment/Train deploys]]
    |who=N/A
    |what=Deploy &lt;code&gt;wmf/1.47.0-wmf.2&lt;/code&gt; to testwikis
}}
{{Deployment calendar event card
    |when=2026-05-11 21:00 SF
    |length=1
    |window=Automatic removal of all obsolete MediaWiki versions from the deployment and bare metal servers (except the most-recent obsolete version)
    |who=N/A
    |what=Runs &lt;code&gt;scap clean auto&lt;/code&gt;
}}
{{Deployment calendar event card
    |when=2026-05-11 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-11 23:00 SF
    |length=0.5
    |window=Primary database switchover
    |who={{ircnick|marostegui|Manuel Arostegui}}, {{ircnick|Amir1|Amir}}, {{ircnick|federico3|Federico Ceratto}}
    |what=Held deployment window for database primary masters maintenance
}}
==={{Deployment_day|date=2026-05-12}}===
{{Deployment calendar event card
    |when=2026-05-12 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-12 01:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.1-&gt;1.47.0-wmf.2|1.47.0-wmf.1|1.47.0-wmf.1}}
* group0 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-12 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-12 05:00 SF
    |length=1
    |window=Mobileapps/RESTBase/Wikifeeds
    |who=Content Transform Team
    |what=Content transform team node services (mobileapps/wikifeeds)
}}
{{Deployment calendar event card
    |when=2026-05-12 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-12 07:00 SF
    |length=0.5
    |window=Test Kitchen UI Deployment Window
    |who=Experimentation Platform Team
    |what=Deployment of Test Kitchen UI (fka MPIC)
}}
{{Deployment calendar event card
    |when=2026-05-12 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-12 08:00 SF
    |length=1
    |window=SRE Collaboration Services office hours
    |who={{ircnick|jelto|Jelto}}, {{ircnick|arnoldokoth|Arnold}}, {{ircnick|mutante|Daniel}}, {{ircnick|arnaudb|Arnaud}}
    |what=Services including Gerrit, Phorge (Phabricator), GitLab
}}
{{Deployment calendar event card
    |when=2026-05-12 09:00 SF
    |length=1
    |window=[[Puppet request window]]&lt;br/&gt;&lt;small&gt;'''(Max 6 patches)'''&lt;/small&gt;
    |who={{ircnick|jhathaway|JHathaway}}, {{ircnick|rzl|Reuven}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to Puppet change''
}}
{{Deployment calendar event card
    |when=2026-05-12 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-12 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version (secondary timeslot)
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.1-&gt;1.47.0-wmf.2|1.47.0-wmf.1|1.47.0-wmf.1}}
* group0 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-12 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-12 14:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-12 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
==={{Deployment_day|date=2026-05-13}}===
{{Deployment calendar event card
    |when=2026-05-13 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-13 01:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.2|1.47.0-wmf.1-&gt;1.47.0-wmf.2|1.47.0-wmf.1}}
* group1 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-13 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-13 04:00 SF
    |length=1
    |window=[[mw:Services|Services]] – [[Citoid]] / [[Zotero]]  
    |who=Marielle ({{ircnick|mvolz}})
    |what=See [[mw:Citoid|Citoid]]
}}
{{Deployment calendar event card
    |when=2026-05-13 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-13 07:00 SF
    |length=1
    |window=Wikifunctions Services UTC Afternoon
    |who=Abstract Wikipedia team (Africa, Europe, Eastern Americas)
    |what=Wikifunctions back-end k8s services
}}
{{Deployment calendar event card
    |when=2026-05-13 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-13 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-13 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version (secondary timeslot)
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.2|1.47.0-wmf.1-&gt;1.47.0-wmf.2|1.47.0-wmf.1}}
* group1 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-13 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-13 14:00 SF
    |length=1
    |window=Wikifunctions Services UTC Late
    |who=Abstract Wikipedia team (North and South America)
    |what=Wikifunctions back-end k8s services
}}
{{Deployment calendar event card
    |when=2026-05-13 15:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-13 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-13 23:00 SF
    |length=0.5
    |window=Primary database switchover
    |who={{ircnick|marostegui|Manuel Arostegui}}, {{ircnick|Amir1|Amir}}, {{ircnick|federico3|Federico Ceratto}}
    |what=Held deployment window for database primary masters maintenance
}}
==={{Deployment_day|date=2026-05-14}}===
{{Deployment calendar event card
    |when=2026-05-14 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-14 01:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.2|1.47.0-wmf.2|1.47.0-wmf.1-&gt;1.47.0-wmf.2}}
* group2 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-14 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-14 05:00 SF
    |length=1
    |window=Mobileapps/RESTBase/Wikifeeds
    |who=Content Transform Team
    |what=Content transform team node services (mobileapps/wikifeeds)
}}
{{Deployment calendar event card
    |when=2026-05-14 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-14 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-14 08:00 SF
    |length=1
    |window=Train log triage
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=See [[Heterogeneous deployment/Train deploys#Breakage]]
}}
{{Deployment calendar event card
    |when=2026-05-14 09:00 SF
    |length=1
    |window=[[Puppet request window]]&lt;br/&gt;&lt;small&gt;'''(Max 6 patches)'''&lt;/small&gt;
    |who={{ircnick|jhathaway|JHathaway}}, {{ircnick|rzl|Reuven}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to Puppet change''
}}
{{Deployment calendar event card
    |when=2026-05-14 10:00 SF
    |length=1
    |window=Cloud Services/Technical Documentation weekly deploy (Toolhub, Developer portal, Striker)
    |who={{ircnick|bd808}}
    |what=...
}}
{{Deployment calendar event card
    |when=2026-05-14 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-14 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version (secondary timeslot)
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.2|1.47.0-wmf.2|1.47.0-wmf.1-&gt;1.47.0-wmf.2}}
* group2 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-14 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-14 14:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-14 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
==={{Deployment_day|date=2026-05-15}}===
{{Deployment calendar event card
    |when=2026-05-15 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}
{{Deployment calendar event card
    |when=2026-05-15 04:00 SF
    |length=0.5
    |window=GitLab version upgrades
    |who={{ircnick|jelto|Jelto}}, {{ircnick|arnoldokoth|Arnold}}, {{ircnick|mutante|Daniel}}, {{ircnick|arnaudb|Arnaud}}
    |what=GitLab version upgrades
}}
==={{Deployment_day|date=2026-05-16}}===
{{Deployment calendar event card
    |when=2026-05-16 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}</text>
      <sha1>7wr3wxfyj6lru6nhvz4ubkr1lh9087h</sha1>
    </revision>
    <revision>
      <id>2408876</id>
      <parentid>2408874</parentid>
      <timestamp>2026-05-04T07:35:33Z</timestamp>
      <contributor>
        <username>1F616EMO</username>
        <id>39448</id>
      </contributor>
      <comment>Undid revision [[Special:diff/2408778|2408778]] by [[Special:Contributions/ScheduleDeploymentBot|ScheduleDeploymentBot]] ([[User talk:ScheduleDeploymentBot|talk]]): rescheduled // [[w:zh:U:1F616EMO/AdvancedRollback|AdvancedRollback]]</comment>
      <origin>2408876</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="43126" sha1="034drb78t59x0f9vg41s9ni59rvbto4" xml:space="preserve">{{Navigation MediaWiki deployment}}
This page tracks '''upcoming''' '''deployments''' of software to the [[:m:Special:SiteMatrix|Wikimedia Foundation servers]].

== Getting started ==
Ensure you joined the {{irc|wikimedia-operations}} IRC channel as all deployment-related communications happen there.
If you need help, contact [[:mw:Wikimedia Release Engineering Team|Release Engineering]] on IRC at {{irc|wikimedia-releng}}; and ping Tyler (&lt;code&gt;thcipriani&lt;/code&gt;).

* '''MediaWiki is deployed weekly''' through the [[/Train|Deployment Train]]. Other services follow their own schedule.
* '''Times are pinned to San Francisco''', thus the UTC time changes in March and November per [[:en:Daylight saving time in the United States|DST]].
* '''Prefer regular [[Backport windows]]''' over adding new windows. To request deployment of a config change or backport, add your username and Gerrit URL to one of the backport windows on this page. You must be online in #wikimedia-operations on IRC during your deployment and install [[WikimediaDebug]] ahead of time. The #wikimedia-operations channel requires you to [[:m:IRC/Instructions#Register your nickname, identify, and enforce|register your nickname]] before you can join.
** You can use the '''backport scheduling tool''' to more easily edit this page: &lt;div style=&quot;text-align: center; margin: 1em 0&quot;&gt;{{Clickable button 2|:toollabs:schedule-deployment|Schedule a backport|class=mw-ui-progressive}}&lt;/div&gt;
* Tasks that meet [[/Inclusion criteria|Inclusion criteria]] '''require their own windows''', which includes long-running tasks. '''Schedule more time''' than you think you need to account for delays and set backs, we recommend one hour for most tasks.
**To create or modify a recurring deploy window, send a patchset to [[:gitlab:repos/releng/release/-/blob/main/make-deployment-calendar/deployments-calendar.yaml|deployments-calendar.yaml file]] in &lt;code&gt;repos/releng/release.git&lt;/code&gt;.
**To create an one-off window, simply edit this page accordingly
** '''Announce''' changes to the [[mail:ops|ops mailing list]] ahead of time if you anticipate or are uncertain about noticeable impacts to database load, HTTP caching, or the introduction of new cookies.
** '''Announce''' deployments of major features to the community via [[:m:Tech/News/Next|Tech News]] and/or via other [[:mw:Wikimedia_Product_Guidance/Communication_channels|Product communication channels]].
* '''Something went wrong?''' See [[Incident response]]. Is there a user-impacting problem? Communicate in the {{irc|wikimedia-operations}} IRC channel. If there is a Phabricator task, ensure [[:phab:tag/wikimedia-incident/|#Wikimedia-Incident]] is tagged, and consider setting the [[:mw:Phabricator/Project_management#Priority_levels|Unbreak Now]] priority.
__TOC__
{{anchor|Next Week|Near Term|Near term|Near-term}}{{clear}}
[[Category:Deployment]]
{{Note|content=Subscribe in Google Calendar via &lt;code&gt;wikimedia.org_rudis09ii2mm5fk4hgdjeh1u64@group.calendar.google.com&lt;/code&gt;.&lt;br&gt;This may not include one-off windows. '''If there are differences, then the wiki page is canonical and correct'''.}}

==Week of May 04==


==={{Deployment_day|date=2026-05-03}}===
{{Deployment calendar event card
    |when=2026-05-03 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}
==={{Deployment_day|date=2026-05-04}}===
{{Deployment calendar event card
    |when=2026-05-04 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|xxb|xxb}}
{{deploy|type=config|gerrit=1279477|title=nlwiki: Modify autoconfirmed requirements for nlwiki|status=}} - {{phabricator|T424898}}
{{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-04 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-04 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|manfredi|manfredi}}
{{deploy|type=1.46.0-wmf.26|gerrit=1281501|title=Email confirmation banner: Remove obsolete arm_b variant|status=}} - {{phabricator|T421366}}
{{deploy|type=1.46.0-wmf.26|gerrit=1281504|title=Use js promise for email confirmation banner|status=}} - {{phabricator|T420007}}
{{ircnick|nya_1F616EMO|1F616EMO}}
{{deploy|type=config|gerrit=1281965|title=zhwikinews: (1/2) revert 20th anniversary logo change (config)|status=}} - {{phabricator|T420165}}
{{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-04 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-04 08:30 SF
    |length=0.5
    |window=Wikimedia Portals Update
    |who={{ircnick|jan_drewniak|Jan Drewniak}}
    |what=Weekly window for the portals page: https://www.wikipedia.org/
}}
{{Deployment calendar event card
    |when=2026-05-04 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-04 10:00 SF
    |length=0.5
    |window=Wikidata Query Service weekly deploy
    |who={{ircnick|ryankemper|Ryan}}
    |what=...
}}
{{Deployment calendar event card
    |when=2026-05-04 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|toyofuku|toyofuku}}
{{deploy|type=config|gerrit=1277667|title=Enable the reading list beta feature survey on all wikipedias|status=}} - {{phabricator|T421776}}
{{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-04 14:00 SF
    |length=2
    |window=Weekly Security deployment window
    |who={{ircnick|Reedy|Sam}}, {{ircnick|sbassett|Scott}}, {{ircnick|Maryum|Maryum}}, {{ircnick|manfredi|Manfredi}}
    |what=Held deployment window for Security-team related deploys.
}}
{{Deployment calendar event card
    |when=2026-05-04 16:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-04 19:00 SF
    |length=1
    |window=Automatic branching of MediaWiki, extensions, skins, and vendor – see [[Heterogeneous deployment/Train deploys]]
    |who=N/A
    |what=Branch &lt;code&gt;wmf/1.47.0-wmf.1&lt;/code&gt;
}}
{{Deployment calendar event card
    |when=2026-05-04 20:00 SF
    |length=1
    |window=Automatic deployment of MediaWiki, extensions, skins, and vendor to testwikis only – see [[Heterogeneous deployment/Train deploys]]
    |who=N/A
    |what=Deploy &lt;code&gt;wmf/1.47.0-wmf.1&lt;/code&gt; to testwikis
}}
{{Deployment calendar event card
    |when=2026-05-04 21:00 SF
    |length=1
    |window=Automatic removal of all obsolete MediaWiki versions from the deployment and bare metal servers (except the most-recent obsolete version)
    |who=N/A
    |what=Runs &lt;code&gt;scap clean auto&lt;/code&gt;
}}
{{Deployment calendar event card
    |when=2026-05-04 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-04 23:00 SF
    |length=0.5
    |window=Primary database switchover
    |who={{ircnick|marostegui|Manuel Arostegui}}, {{ircnick|Amir1|Amir}}, {{ircnick|federico3|Federico Ceratto}}
    |what=Held deployment window for database primary masters maintenance
}}
==={{Deployment_day|date=2026-05-05}}===
{{Deployment calendar event card
    |when=2026-05-05 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-05 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-05 05:00 SF
    |length=1
    |window=Mobileapps/RESTBase/Wikifeeds
    |who=Content Transform Team
    |what=Content transform team node services (mobileapps/wikifeeds)
}}
{{Deployment calendar event card
    |when=2026-05-05 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|MatmaRex|Bartosz}}
{{deploy|type=config|gerrit=1270882|title=Remove temporary `wgOAuth2UsePrefixedSub` feature flag|status=}} - {{phabricator|T417690}}
{{deploy|type=config|gerrit=1271969|title=Move privileged global and local group handling to WikimediaCustomizations|status=}} - {{phabricator|T418507}}
{{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-05 07:00 SF
    |length=0.5
    |window=Test Kitchen UI Deployment Window
    |who=Experimentation Platform Team
    |what=Deployment of Test Kitchen UI (fka MPIC)
}}
{{Deployment calendar event card
    |when=2026-05-05 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-05 08:00 SF
    |length=1
    |window=SRE Collaboration Services office hours
    |who={{ircnick|jelto|Jelto}}, {{ircnick|arnoldokoth|Arnold}}, {{ircnick|mutante|Daniel}}, {{ircnick|arnaudb|Arnaud}}
    |what=Services including Gerrit, Phorge (Phabricator), GitLab
}}
{{Deployment calendar event card
    |when=2026-05-05 09:00 SF
    |length=1
    |window=[[Puppet request window]]&lt;br/&gt;&lt;small&gt;'''(Max 6 patches)'''&lt;/small&gt;
    |who={{ircnick|jhathaway|JHathaway}}, {{ircnick|rzl|Reuven}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to Puppet change''
}}
{{Deployment calendar event card
    |when=2026-05-05 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-05 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-7 Version
    |who={{ircnick|brennen|Brennen}}, {{ircnick|jeena|Jeena}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.46.0-wmf.26-&gt;1.47.0-wmf.1|1.46.0-wmf.26|1.46.0-wmf.26}}
* group0 to [[mw:MediaWiki_1.47/wmf.1|1.47.0-wmf.1]]
* '''Blockers: {{phabricator|T423910}}'''
}}
{{Deployment calendar event card
    |when=2026-05-05 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-05 14:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-05 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
==={{Deployment_day|date=2026-05-06}}===
{{Deployment calendar event card
    |when=2026-05-06 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-06 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-06 04:00 SF
    |length=1
    |window=[[mw:Services|Services]] – [[Citoid]] / [[Zotero]]
    |who=Marielle ({{ircnick|mvolz}})
    |what=See [[mw:Citoid|Citoid]]
}}
{{Deployment calendar event card
    |when=2026-05-06 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-06 07:00 SF
    |length=1
    |window=Wikifunctions Services UTC Afternoon
    |who=Abstract Wikipedia team (Africa, Europe, Eastern Americas)
    |what=Wikifunctions back-end k8s services
}}
{{Deployment calendar event card
    |when=2026-05-06 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-06 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-06 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-7 Version
    |who={{ircnick|brennen|Brennen}}, {{ircnick|jeena|Jeena}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.1|1.46.0-wmf.26-&gt;1.47.0-wmf.1|1.46.0-wmf.26}}
* group1 to [[mw:MediaWiki_1.47/wmf.1|1.47.0-wmf.1]]
* '''Blockers: {{phabricator|T423910}}'''
}}
{{Deployment calendar event card
    |when=2026-05-06 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-06 14:00 SF
    |length=1
    |window=Wikifunctions Services UTC Late
    |who=Abstract Wikipedia team (North and South America)
    |what=Wikifunctions back-end k8s services
}}
{{Deployment calendar event card
    |when=2026-05-06 15:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-06 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-06 23:00 SF
    |length=0.5
    |window=Primary database switchover
    |who={{ircnick|marostegui|Manuel Arostegui}}, {{ircnick|Amir1|Amir}}, {{ircnick|federico3|Federico Ceratto}}
    |what=Held deployment window for database primary masters maintenance
}}
==={{Deployment_day|date=2026-05-07}}===
{{Deployment calendar event card
    |when=2026-05-07 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-07 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-07 05:00 SF
    |length=1
    |window=Mobileapps/RESTBase/Wikifeeds
    |who=Content Transform Team
    |what=Content transform team node services (mobileapps/wikifeeds)
}}
{{Deployment calendar event card
    |when=2026-05-07 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-07 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-07 08:00 SF
    |length=1
    |window=Train log triage
    |who={{ircnick|brennen|Brennen}}, {{ircnick|jeena|Jeena}}
    |what=See [[Heterogeneous deployment/Train deploys#Breakage]]
}}
{{Deployment calendar event card
    |when=2026-05-07 09:00 SF
    |length=1
    |window=[[Puppet request window]]&lt;br/&gt;&lt;small&gt;'''(Max 6 patches)'''&lt;/small&gt;
    |who={{ircnick|jhathaway|JHathaway}}, {{ircnick|rzl|Reuven}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to Puppet change''
}}
{{Deployment calendar event card
    |when=2026-05-07 10:00 SF
    |length=1
    |window=Cloud Services/Technical Documentation weekly deploy (Toolhub, Developer portal, Striker)
    |who={{ircnick|bd808}}
    |what=...
}}
{{Deployment calendar event card
    |when=2026-05-07 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-07 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-7 Version
    |who={{ircnick|brennen|Brennen}}, {{ircnick|jeena|Jeena}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.1|1.47.0-wmf.1|1.46.0-wmf.26-&gt;1.47.0-wmf.1}}
* group2 to [[mw:MediaWiki_1.47/wmf.1|1.47.0-wmf.1]]
* '''Blockers: {{phabricator|T423910}}'''
}}
{{Deployment calendar event card
    |when=2026-05-07 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-07 14:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-07 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
==={{Deployment_day|date=2026-05-08}}===
{{Deployment calendar event card
    |when=2026-05-08 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}
{{Deployment calendar event card
    |when=2026-05-08 04:00 SF
    |length=0.5
    |window=GitLab version upgrades
    |who={{ircnick|jelto|Jelto}}, {{ircnick|arnoldokoth|Arnold}}, {{ircnick|mutante|Daniel}}, {{ircnick|arnaudb|Arnaud}}
    |what=GitLab version upgrades
}}
==={{Deployment_day|date=2026-05-09}}===
{{Deployment calendar event card
    |when=2026-05-09 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}

==Week of May 11==


==={{Deployment_day|date=2026-05-10}}===
{{Deployment calendar event card
    |when=2026-05-10 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}
==={{Deployment_day|date=2026-05-11}}===
{{Deployment calendar event card
    |when=2026-05-11 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-11 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-11 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-11 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-11 08:30 SF
    |length=0.5
    |window=Wikimedia Portals Update
    |who={{ircnick|jan_drewniak|Jan Drewniak}}
    |what=Weekly window for the portals page: https://www.wikipedia.org/
}}
{{Deployment calendar event card
    |when=2026-05-11 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-11 10:00 SF
    |length=0.5
    |window=Wikidata Query Service weekly deploy
    |who={{ircnick|ryankemper|Ryan}}
    |what=...
}}
{{Deployment calendar event card
    |when=2026-05-11 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-11 14:00 SF
    |length=2
    |window=Weekly Security deployment window
    |who={{ircnick|Reedy|Sam}}, {{ircnick|sbassett|Scott}}, {{ircnick|Maryum|Maryum}}, {{ircnick|manfredi|Manfredi}}
    |what=Held deployment window for Security-team related deploys.
}}
{{Deployment calendar event card
    |when=2026-05-11 16:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-11 19:00 SF
    |length=1
    |window=Automatic branching of MediaWiki, extensions, skins, and vendor – see [[Heterogeneous deployment/Train deploys]]
    |who=N/A
    |what=Branch &lt;code&gt;wmf/1.47.0-wmf.2&lt;/code&gt;
}}
{{Deployment calendar event card
    |when=2026-05-11 20:00 SF
    |length=1
    |window=Automatic deployment of MediaWiki, extensions, skins, and vendor to testwikis only – see [[Heterogeneous deployment/Train deploys]]
    |who=N/A
    |what=Deploy &lt;code&gt;wmf/1.47.0-wmf.2&lt;/code&gt; to testwikis
}}
{{Deployment calendar event card
    |when=2026-05-11 21:00 SF
    |length=1
    |window=Automatic removal of all obsolete MediaWiki versions from the deployment and bare metal servers (except the most-recent obsolete version)
    |who=N/A
    |what=Runs &lt;code&gt;scap clean auto&lt;/code&gt;
}}
{{Deployment calendar event card
    |when=2026-05-11 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-11 23:00 SF
    |length=0.5
    |window=Primary database switchover
    |who={{ircnick|marostegui|Manuel Arostegui}}, {{ircnick|Amir1|Amir}}, {{ircnick|federico3|Federico Ceratto}}
    |what=Held deployment window for database primary masters maintenance
}}
==={{Deployment_day|date=2026-05-12}}===
{{Deployment calendar event card
    |when=2026-05-12 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-12 01:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.1-&gt;1.47.0-wmf.2|1.47.0-wmf.1|1.47.0-wmf.1}}
* group0 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-12 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-12 05:00 SF
    |length=1
    |window=Mobileapps/RESTBase/Wikifeeds
    |who=Content Transform Team
    |what=Content transform team node services (mobileapps/wikifeeds)
}}
{{Deployment calendar event card
    |when=2026-05-12 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-12 07:00 SF
    |length=0.5
    |window=Test Kitchen UI Deployment Window
    |who=Experimentation Platform Team
    |what=Deployment of Test Kitchen UI (fka MPIC)
}}
{{Deployment calendar event card
    |when=2026-05-12 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-12 08:00 SF
    |length=1
    |window=SRE Collaboration Services office hours
    |who={{ircnick|jelto|Jelto}}, {{ircnick|arnoldokoth|Arnold}}, {{ircnick|mutante|Daniel}}, {{ircnick|arnaudb|Arnaud}}
    |what=Services including Gerrit, Phorge (Phabricator), GitLab
}}
{{Deployment calendar event card
    |when=2026-05-12 09:00 SF
    |length=1
    |window=[[Puppet request window]]&lt;br/&gt;&lt;small&gt;'''(Max 6 patches)'''&lt;/small&gt;
    |who={{ircnick|jhathaway|JHathaway}}, {{ircnick|rzl|Reuven}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to Puppet change''
}}
{{Deployment calendar event card
    |when=2026-05-12 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-12 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version (secondary timeslot)
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.1-&gt;1.47.0-wmf.2|1.47.0-wmf.1|1.47.0-wmf.1}}
* group0 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-12 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-12 14:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-12 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
==={{Deployment_day|date=2026-05-13}}===
{{Deployment calendar event card
    |when=2026-05-13 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-13 01:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.2|1.47.0-wmf.1-&gt;1.47.0-wmf.2|1.47.0-wmf.1}}
* group1 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-13 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-13 04:00 SF
    |length=1
    |window=[[mw:Services|Services]] – [[Citoid]] / [[Zotero]]  
    |who=Marielle ({{ircnick|mvolz}})
    |what=See [[mw:Citoid|Citoid]]
}}
{{Deployment calendar event card
    |when=2026-05-13 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-13 07:00 SF
    |length=1
    |window=Wikifunctions Services UTC Afternoon
    |who=Abstract Wikipedia team (Africa, Europe, Eastern Americas)
    |what=Wikifunctions back-end k8s services
}}
{{Deployment calendar event card
    |when=2026-05-13 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-13 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-13 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version (secondary timeslot)
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.2|1.47.0-wmf.1-&gt;1.47.0-wmf.2|1.47.0-wmf.1}}
* group1 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-13 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-13 14:00 SF
    |length=1
    |window=Wikifunctions Services UTC Late
    |who=Abstract Wikipedia team (North and South America)
    |what=Wikifunctions back-end k8s services
}}
{{Deployment calendar event card
    |when=2026-05-13 15:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-13 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-13 23:00 SF
    |length=0.5
    |window=Primary database switchover
    |who={{ircnick|marostegui|Manuel Arostegui}}, {{ircnick|Amir1|Amir}}, {{ircnick|federico3|Federico Ceratto}}
    |what=Held deployment window for database primary masters maintenance
}}
==={{Deployment_day|date=2026-05-14}}===
{{Deployment calendar event card
    |when=2026-05-14 00:00 SF
    |length=1
    |window=[[Backport windows|UTC morning backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Amir1|Amir}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|awight|Adam}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-14 01:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.2|1.47.0-wmf.2|1.47.0-wmf.1-&gt;1.47.0-wmf.2}}
* group2 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-14 03:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC mid-day)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-14 05:00 SF
    |length=1
    |window=Mobileapps/RESTBase/Wikifeeds
    |who=Content Transform Team
    |what=Content transform team node services (mobileapps/wikifeeds)
}}
{{Deployment calendar event card
    |when=2026-05-14 06:00 SF
    |length=1
    |window=[[Backport windows|UTC afternoon backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|Lucas_WMDE|Lucas}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-14 07:30 SF
    |length=0.5
    |window=Test Kitchen Experiment Deployment Window
    |who=Test Kitchen
    |what=Automatic start/stop of active experiments and instruments managed by [[Test Kitchen]].
}}
{{Deployment calendar event card
    |when=2026-05-14 08:00 SF
    |length=1
    |window=Train log triage
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=See [[Heterogeneous deployment/Train deploys#Breakage]]
}}
{{Deployment calendar event card
    |when=2026-05-14 09:00 SF
    |length=1
    |window=[[Puppet request window]]&lt;br/&gt;&lt;small&gt;'''(Max 6 patches)'''&lt;/small&gt;
    |who={{ircnick|jhathaway|JHathaway}}, {{ircnick|rzl|Reuven}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to Puppet change''
}}
{{Deployment calendar event card
    |when=2026-05-14 10:00 SF
    |length=1
    |window=Cloud Services/Technical Documentation weekly deploy (Toolhub, Developer portal, Striker)
    |who={{ircnick|bd808}}
    |what=...
}}
{{Deployment calendar event card
    |when=2026-05-14 10:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC late)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
{{Deployment calendar event card
    |when=2026-05-14 11:00 SF
    |length=2
    |window=MediaWiki train - Utc-0+Utc-7 Version (secondary timeslot)
    |who={{ircnick|andre|Andre}}, {{ircnick|brennen|Brennen}}
    |what=[[mw:MediaWiki 1.47/Roadmap#Schedule for the deployments|1.47 schedule]]
{{DeployOneWeekMini|1.47.0-wmf.2|1.47.0-wmf.2|1.47.0-wmf.1-&gt;1.47.0-wmf.2}}
* group2 to [[mw:MediaWiki_1.47/wmf.2|1.47.0-wmf.2]]
* '''Blockers: {{phabricator|T423911}}'''
}}
{{Deployment calendar event card
    |when=2026-05-14 13:00 SF
    |length=1
    |window=[[Backport windows|UTC late backport window]]&lt;br/&gt;&lt;small&gt;'''Your patch may or may not be deployed at the sole discretion of the deployer'''&lt;/small&gt;
    |who={{ircnick|RoanKattouw|Roan}}, {{ircnick|Urbanecm|Martin}}, {{ircnick|TheresNoTime|Sammy}}, {{ircnick|kindrobot|Stef}}, {{ircnick|cjming|Clare}}
    |what={{ircnick|irc-nickname|Requesting Developer}}
* ''Gerrit link to backport or config change''
}}
{{Deployment calendar event card
    |when=2026-05-14 14:00 SF
    |length=1
    |window=Readers deployment window
    |who=Readers
    |what=NOTE: often skipped, the reader teams do not typically check IRC so assume this is not being used if 5 minutes past the start
}}
{{Deployment calendar event card
    |when=2026-05-14 23:00 SF
    |length=1
    |window=[[MediaWiki_On_Kubernetes#How_to_manage_changes_to_the_infrastructure|MediaWiki infrastructure]] (UTC early)
    |who=SRE team
    |what=MediaWiki-related infrastructure changes that need a kubernetes deployment.
}}
==={{Deployment_day|date=2026-05-15}}===
{{Deployment calendar event card
    |when=2026-05-15 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}
{{Deployment calendar event card
    |when=2026-05-15 04:00 SF
    |length=0.5
    |window=GitLab version upgrades
    |who={{ircnick|jelto|Jelto}}, {{ircnick|arnoldokoth|Arnold}}, {{ircnick|mutante|Daniel}}, {{ircnick|arnaudb|Arnaud}}
    |what=GitLab version upgrades
}}
==={{Deployment_day|date=2026-05-16}}===
{{Deployment calendar event card
    |when=2026-05-16 00:00 SF
    |length=24
    |window=No deploys all day! See [[Deployments/Emergencies]] if things are broken.
    |who=
    |what=No Deploys
}}</text>
      <sha1>034drb78t59x0f9vg41s9ni59rvbto4</sha1>
    </revision>
  </page>
  <page>
    <title>Server Admin Log</title>
    <ns>0</ns>
    <id>7919</id>
    <revision>
      <id>2408797</id>
      <parentid>2408779</parentid>
      <timestamp>2026-05-03T12:27:37Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963|Remove Wikinews from installer's default main page]] (duration: 29m 22s)</comment>
      <origin>2408797</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="48838" sha1="ge1fqnusqs320bspsi92js5ll1qsml3" xml:space="preserve">
== 2026-05-03 ==
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ge1fqnusqs320bspsi92js5ll1qsml3</sha1>
    </revision>
    <revision>
      <id>2408824</id>
      <parentid>2408797</parentid>
      <timestamp>2026-05-03T14:00:43Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991|Disable uploads in scnwiki (T425278)]]</comment>
      <origin>2408824</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="48966" sha1="lj3cdhlq6tabpik53b930yb6y6vpe94" xml:space="preserve">
== 2026-05-03 ==
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>lj3cdhlq6tabpik53b930yb6y6vpe94</sha1>
    </revision>
    <revision>
      <id>2408825</id>
      <parentid>2408824</parentid>
      <timestamp>2026-05-03T14:04:39Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991|Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.</comment>
      <origin>2408825</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="49192" sha1="el7bmhrrb4gih8ujttcbisa6dcjo2ap" xml:space="preserve">
== 2026-05-03 ==
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>el7bmhrrb4gih8ujttcbisa6dcjo2ap</sha1>
    </revision>
    <revision>
      <id>2408826</id>
      <parentid>2408825</parentid>
      <timestamp>2026-05-03T14:05:11Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>ladsgroup@deploy1003: ladsgroup: Continuing with deployment</comment>
      <origin>2408826</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="49260" sha1="k9bbgmdurxwtl0zlhtnqrke0bqf6ucp" xml:space="preserve">
== 2026-05-03 ==
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>k9bbgmdurxwtl0zlhtnqrke0bqf6ucp</sha1>
    </revision>
    <revision>
      <id>2408827</id>
      <parentid>2408826</parentid>
      <timestamp>2026-05-03T14:11:34Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991|Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)</comment>
      <origin>2408827</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="49409" sha1="7mdvulmhxd821tikervp4wwhdpmr6cy" xml:space="preserve">
== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>7mdvulmhxd821tikervp4wwhdpmr6cy</sha1>
    </revision>
    <revision>
      <id>2408838</id>
      <parentid>2408827</parentid>
      <timestamp>2026-05-04T02:00:42Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image</comment>
      <origin>2408838</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="49510" sha1="asq2xkk2x0yqtdbtj16xnh0tr38rsf3" xml:space="preserve">
== 2026-05-04 ==
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>asq2xkk2x0yqtdbtj16xnh0tr38rsf3</sha1>
    </revision>
    <revision>
      <id>2408839</id>
      <parentid>2408838</parentid>
      <timestamp>2026-05-04T02:07:18Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)</comment>
      <origin>2408839</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="49614" sha1="bf7jk1xvqt62lmfb5qhc5ia6d6j1tme" xml:space="preserve">
== 2026-05-04 ==
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>bf7jk1xvqt62lmfb5qhc5ia6d6j1tme</sha1>
    </revision>
    <revision>
      <id>2408841</id>
      <parentid>2408839</parentid>
      <timestamp>2026-05-04T05:54:58Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie</comment>
      <origin>2408841</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="49765" sha1="hhf4hprc4jdsz2fskd87m94602ujbrp" xml:space="preserve">
== 2026-05-04 ==
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>hhf4hprc4jdsz2fskd87m94602ujbrp</sha1>
    </revision>
    <revision>
      <id>2408842</id>
      <parentid>2408841</parentid>
      <timestamp>2026-05-04T05:55:04Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie</comment>
      <origin>2408842</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="49862" sha1="1rxde99d6ujipoi7u5fn5oggsxo3ah2" xml:space="preserve">
== 2026-05-04 ==
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>1rxde99d6ujipoi7u5fn5oggsxo3ah2</sha1>
    </revision>
    <revision>
      <id>2408843</id>
      <parentid>2408842</parentid>
      <timestamp>2026-05-04T05:55:42Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie</comment>
      <origin>2408843</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="49978" sha1="9mua4xsnpia0cdezugtzwe8exuq7qb2" xml:space="preserve">
== 2026-05-04 ==
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>9mua4xsnpia0cdezugtzwe8exuq7qb2</sha1>
    </revision>
    <revision>
      <id>2408844</id>
      <parentid>2408843</parentid>
      <timestamp>2026-05-04T05:57:10Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie</comment>
      <origin>2408844</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="50086" sha1="dxo4da8gqpvcz1x81p4hwjwjv9fq2jq" xml:space="preserve">
== 2026-05-04 ==
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>dxo4da8gqpvcz1x81p4hwjwjv9fq2jq</sha1>
    </revision>
    <revision>
      <id>2408845</id>
      <parentid>2408844</parentid>
      <timestamp>2026-05-04T05:58:36Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie</comment>
      <origin>2408845</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="50237" sha1="6m6rmlid79z1vljncyet0vmakeoxhk6" xml:space="preserve">
== 2026-05-04 ==
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>6m6rmlid79z1vljncyet0vmakeoxhk6</sha1>
    </revision>
    <revision>
      <id>2408846</id>
      <parentid>2408845</parentid>
      <timestamp>2026-05-04T05:58:41Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie</comment>
      <origin>2408846</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="50334" sha1="2pt7bxvr2au6lld9t3vbm3hnko1d26q" xml:space="preserve">
== 2026-05-04 ==
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>2pt7bxvr2au6lld9t3vbm3hnko1d26q</sha1>
    </revision>
    <revision>
      <id>2408847</id>
      <parentid>2408846</parentid>
      <timestamp>2026-05-04T06:02:50Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie</comment>
      <origin>2408847</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="50450" sha1="ar2b1guke6vzmv8dii2cqdj6vdixdgf" xml:space="preserve">
== 2026-05-04 ==
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ar2b1guke6vzmv8dii2cqdj6vdixdgf</sha1>
    </revision>
    <revision>
      <id>2408848</id>
      <parentid>2408847</parentid>
      <timestamp>2026-05-04T06:05:32Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie</comment>
      <origin>2408848</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="50558" sha1="74ujxsfyex8rck9ofdyk2vlrbbf5vtt" xml:space="preserve">
== 2026-05-04 ==
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>74ujxsfyex8rck9ofdyk2vlrbbf5vtt</sha1>
    </revision>
    <revision>
      <id>2408849</id>
      <parentid>2408848</parentid>
      <timestamp>2026-05-04T06:09:37Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 T424792</comment>
      <origin>2408849</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="50679" sha1="funqgf2mizai2k85u5znwwl9zxc5bcv" xml:space="preserve">
== 2026-05-04 ==
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>funqgf2mizai2k85u5znwwl9zxc5bcv</sha1>
    </revision>
    <revision>
      <id>2408850</id>
      <parentid>2408849</parentid>
      <timestamp>2026-05-04T06:10:29Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie</comment>
      <origin>2408850</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="50849" sha1="8339l24846hw4g18hn0wh8jb4ddksxx" xml:space="preserve">
== 2026-05-04 ==
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>8339l24846hw4g18hn0wh8jb4ddksxx</sha1>
    </revision>
    <revision>
      <id>2408851</id>
      <parentid>2408850</parentid>
      <timestamp>2026-05-04T06:11:18Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie</comment>
      <origin>2408851</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="51000" sha1="lzoeln2mx6233v1jfl6evzk5vjmx1et" xml:space="preserve">
== 2026-05-04 ==
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>lzoeln2mx6233v1jfl6evzk5vjmx1et</sha1>
    </revision>
    <revision>
      <id>2408852</id>
      <parentid>2408851</parentid>
      <timestamp>2026-05-04T06:11:24Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie</comment>
      <origin>2408852</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="51097" sha1="cpr8u103cvha5x6w560yj8gjtkjxtgh" xml:space="preserve">
== 2026-05-04 ==
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>cpr8u103cvha5x6w560yj8gjtkjxtgh</sha1>
    </revision>
    <revision>
      <id>2408853</id>
      <parentid>2408852</parentid>
      <timestamp>2026-05-04T06:11:41Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie</comment>
      <origin>2408853</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="51213" sha1="66q01t6fo0s679q21elxeqx8si89b3p" xml:space="preserve">
== 2026-05-04 ==
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>66q01t6fo0s679q21elxeqx8si89b3p</sha1>
    </revision>
    <revision>
      <id>2408854</id>
      <parentid>2408853</parentid>
      <timestamp>2026-05-04T06:17:58Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage</comment>
      <origin>2408854</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="51339" sha1="f1f5oqz5bquzxca0f2xqcn5uftxobho" xml:space="preserve">
== 2026-05-04 ==
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>f1f5oqz5bquzxca0f2xqcn5uftxobho</sha1>
    </revision>
    <revision>
      <id>2408855</id>
      <parentid>2408854</parentid>
      <timestamp>2026-05-04T06:19:13Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage</comment>
      <origin>2408855</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="51465" sha1="iaemr25zxe0ujn0263rm1z7mpl2ruox" xml:space="preserve">
== 2026-05-04 ==
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>iaemr25zxe0ujn0263rm1z7mpl2ruox</sha1>
    </revision>
    <revision>
      <id>2408856</id>
      <parentid>2408855</parentid>
      <timestamp>2026-05-04T06:21:50Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie</comment>
      <origin>2408856</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="51573" sha1="7yivqtg8n2cixus1irj0imc7ochav7l" xml:space="preserve">
== 2026-05-04 ==
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>7yivqtg8n2cixus1irj0imc7ochav7l</sha1>
    </revision>
    <revision>
      <id>2408857</id>
      <parentid>2408856</parentid>
      <timestamp>2026-05-04T06:25:26Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage</comment>
      <origin>2408857</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="51718" sha1="rbq1b0ntoi33pjfxfiip94n5qjhdez9" xml:space="preserve">
== 2026-05-04 ==
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>rbq1b0ntoi33pjfxfiip94n5qjhdez9</sha1>
    </revision>
    <revision>
      <id>2408858</id>
      <parentid>2408857</parentid>
      <timestamp>2026-05-04T06:29:40Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage</comment>
      <origin>2408858</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="51863" sha1="1pzu3n0m9nvexbzxenhvm0l99hekzxc" xml:space="preserve">
== 2026-05-04 ==
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>1pzu3n0m9nvexbzxenhvm0l99hekzxc</sha1>
    </revision>
    <revision>
      <id>2408859</id>
      <parentid>2408858</parentid>
      <timestamp>2026-05-04T06:37:05Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage</comment>
      <origin>2408859</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="51989" sha1="te3e8ynn250ywh0i8f6ty03ayj58dmk" xml:space="preserve">
== 2026-05-04 ==
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>te3e8ynn250ywh0i8f6ty03ayj58dmk</sha1>
    </revision>
    <revision>
      <id>2408860</id>
      <parentid>2408859</parentid>
      <timestamp>2026-05-04T06:43:29Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage</comment>
      <origin>2408860</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="52134" sha1="bozn2fftzkhiugu0xb417i2u0h8p2ih" xml:space="preserve">
== 2026-05-04 ==
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>bozn2fftzkhiugu0xb417i2u0h8p2ih</sha1>
    </revision>
    <revision>
      <id>2408861</id>
      <parentid>2408860</parentid>
      <timestamp>2026-05-04T06:47:36Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie</comment>
      <origin>2408861</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="52261" sha1="58dwxg8hrowomq1kvszahq7304dh94e" xml:space="preserve">
== 2026-05-04 ==
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>58dwxg8hrowomq1kvszahq7304dh94e</sha1>
    </revision>
    <revision>
      <id>2408862</id>
      <parentid>2408861</parentid>
      <timestamp>2026-05-04T06:52:15Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie</comment>
      <origin>2408862</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="52388" sha1="6se89gpofzncf9ff1r7b3itdzi30z15" xml:space="preserve">
== 2026-05-04 ==
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>6se89gpofzncf9ff1r7b3itdzi30z15</sha1>
    </revision>
    <revision>
      <id>2408863</id>
      <parentid>2408862</parentid>
      <timestamp>2026-05-04T06:55:05Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie</comment>
      <origin>2408863</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="52487" sha1="mjk9iv8dgi5rfhxfgckt9ec4rejrqnk" xml:space="preserve">
== 2026-05-04 ==
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>mjk9iv8dgi5rfhxfgckt9ec4rejrqnk</sha1>
    </revision>
    <revision>
      <id>2408864</id>
      <parentid>2408863</parentid>
      <timestamp>2026-05-04T06:56:01Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie</comment>
      <origin>2408864</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="52586" sha1="oqysguygbhg1hizxdw9657irw42lk8d" xml:space="preserve">
== 2026-05-04 ==
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>oqysguygbhg1hizxdw9657irw42lk8d</sha1>
    </revision>
    <revision>
      <id>2408865</id>
      <parentid>2408864</parentid>
      <timestamp>2026-05-04T07:05:22Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie</comment>
      <origin>2408865</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="52713" sha1="01qebei4twnw3k244eavdr2y5ysje10" xml:space="preserve">
== 2026-05-04 ==
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>01qebei4twnw3k244eavdr2y5ysje10</sha1>
    </revision>
    <revision>
      <id>2408866</id>
      <parentid>2408865</parentid>
      <timestamp>2026-05-04T07:11:53Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie</comment>
      <origin>2408866</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="52812" sha1="gknyr9ia5hvrsmrpuu1ugm6ct8a9rlo" xml:space="preserve">
== 2026-05-04 ==
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>gknyr9ia5hvrsmrpuu1ugm6ct8a9rlo</sha1>
    </revision>
    <revision>
      <id>2408867</id>
      <parentid>2408866</parentid>
      <timestamp>2026-05-04T07:16:03Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org</comment>
      <origin>2408867</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="52912" sha1="leool1eggd1ho8sakcdm6cx9kdzenrn" xml:space="preserve">
== 2026-05-04 ==
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>leool1eggd1ho8sakcdm6cx9kdzenrn</sha1>
    </revision>
    <revision>
      <id>2408868</id>
      <parentid>2408867</parentid>
      <timestamp>2026-05-04T07:20:06Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org</comment>
      <origin>2408868</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="53031" sha1="8qeuyn2f2exgax07q82gp4785ctbr3n" xml:space="preserve">
== 2026-05-04 ==
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>8qeuyn2f2exgax07q82gp4785ctbr3n</sha1>
    </revision>
    <revision>
      <id>2408871</id>
      <parentid>2408868</parentid>
      <timestamp>2026-05-04T07:28:10Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet</comment>
      <origin>2408871</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="53130" sha1="ralz1xbi0rm0i9fqx7jtupr88xxtlw3" xml:space="preserve">
== 2026-05-04 ==
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ralz1xbi0rm0i9fqx7jtupr88xxtlw3</sha1>
    </revision>
    <revision>
      <id>2408872</id>
      <parentid>2408871</parentid>
      <timestamp>2026-05-04T07:33:02Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.dns.netbox</comment>
      <origin>2408872</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="53192" sha1="22uylii37vaotb1sxst0744s99o37sm" xml:space="preserve">
== 2026-05-04 ==
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>22uylii37vaotb1sxst0744s99o37sm</sha1>
    </revision>
    <revision>
      <id>2408873</id>
      <parentid>2408872</parentid>
      <timestamp>2026-05-04T07:33:04Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet</comment>
      <origin>2408873</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="53294" sha1="bahpu2sucamyy51j588q4tn0qt0o5d5" xml:space="preserve">
== 2026-05-04 ==
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>bahpu2sucamyy51j588q4tn0qt0o5d5</sha1>
    </revision>
    <revision>
      <id>2408875</id>
      <parentid>2408873</parentid>
      <timestamp>2026-05-04T07:35:30Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply</comment>
      <origin>2408875</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="53428" sha1="n1s9zc5nrvb5nwqep0ye31oq71lt7yb" xml:space="preserve">
== 2026-05-04 ==
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>n1s9zc5nrvb5nwqep0ye31oq71lt7yb</sha1>
    </revision>
    <revision>
      <id>2408877</id>
      <parentid>2408875</parentid>
      <timestamp>2026-05-04T07:35:39Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply</comment>
      <origin>2408877</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="53561" sha1="nttmj3v9leg0k54jz3bnf3syyn8rtb1" xml:space="preserve">
== 2026-05-04 ==
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>nttmj3v9leg0k54jz3bnf3syyn8rtb1</sha1>
    </revision>
    <revision>
      <id>2408878</id>
      <parentid>2408877</parentid>
      <timestamp>2026-05-04T07:37:01Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet</comment>
      <origin>2408878</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="53682" sha1="q6jmlblpjgyb3mbnzyfil6noim38uvg" xml:space="preserve">
== 2026-05-04 ==
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>q6jmlblpjgyb3mbnzyfil6noim38uvg</sha1>
    </revision>
    <revision>
      <id>2408879</id>
      <parentid>2408878</parentid>
      <timestamp>2026-05-04T07:38:46Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>moritzm: installing Linux 6.12.85 on trixie hosts</comment>
      <origin>2408879</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="53740" sha1="ol5gwqgo7z8pjkp5kmsiqaw27mdivwj" xml:space="preserve">
== 2026-05-04 ==
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ol5gwqgo7z8pjkp5kmsiqaw27mdivwj</sha1>
    </revision>
    <revision>
      <id>2408880</id>
      <parentid>2408879</parentid>
      <timestamp>2026-05-04T07:40:31Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie</comment>
      <origin>2408880</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="53858" sha1="rj7h7mf3mghfaal0fvn6v0ryqo1nhl8" xml:space="preserve">
== 2026-05-04 ==
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>rj7h7mf3mghfaal0fvn6v0ryqo1nhl8</sha1>
    </revision>
    <revision>
      <id>2408881</id>
      <parentid>2408880</parentid>
      <timestamp>2026-05-04T07:41:26Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie</comment>
      <origin>2408881</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="53976" sha1="7wnc4vtrrqi9qg5qxjul0d2n7i92o3r" xml:space="preserve">
== 2026-05-04 ==
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>7wnc4vtrrqi9qg5qxjul0d2n7i92o3r</sha1>
    </revision>
    <revision>
      <id>2408882</id>
      <parentid>2408881</parentid>
      <timestamp>2026-05-04T07:42:32Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;</comment>
      <origin>2408882</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="54221" sha1="8fco5b83lgo0i7oe13c6slhg7bjmojy" xml:space="preserve">
== 2026-05-04 ==
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>8fco5b83lgo0i7oe13c6slhg7bjmojy</sha1>
    </revision>
    <revision>
      <id>2408883</id>
      <parentid>2408882</parentid>
      <timestamp>2026-05-04T07:42:48Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;</comment>
      <origin>2408883</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="54485" sha1="s2xr8zqa37j3ak4zshaf2hir6b88nvw" xml:space="preserve">
== 2026-05-04 ==
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>s2xr8zqa37j3ak4zshaf2hir6b88nvw</sha1>
    </revision>
    <revision>
      <id>2408884</id>
      <parentid>2408883</parentid>
      <timestamp>2026-05-04T07:42:49Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)</comment>
      <origin>2408884</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="54566" sha1="r5n40n1p0dey5p5fdpglp2ue3m974lq" xml:space="preserve">
== 2026-05-04 ==
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>r5n40n1p0dey5p5fdpglp2ue3m974lq</sha1>
    </revision>
    <revision>
      <id>2408885</id>
      <parentid>2408884</parentid>
      <timestamp>2026-05-04T07:42:49Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet</comment>
      <origin>2408885</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="54684" sha1="60dle7dxisd1euwuqowk38rux1nix7z" xml:space="preserve">
== 2026-05-04 ==
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>60dle7dxisd1euwuqowk38rux1nix7z</sha1>
    </revision>
    <revision>
      <id>2408886</id>
      <parentid>2408885</parentid>
      <timestamp>2026-05-04T07:44:12Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply</comment>
      <origin>2408886</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="54787" sha1="c108hdgkl98o9zexe8h3ax1nc174ie2" xml:space="preserve">
== 2026-05-04 ==
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>c108hdgkl98o9zexe8h3ax1nc174ie2</sha1>
    </revision>
    <revision>
      <id>2408887</id>
      <parentid>2408886</parentid>
      <timestamp>2026-05-04T07:44:19Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply</comment>
      <origin>2408887</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="54889" sha1="itk3oqy31bzl8rggwo9s0wo1sb33jxu" xml:space="preserve">
== 2026-05-04 ==
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>itk3oqy31bzl8rggwo9s0wo1sb33jxu</sha1>
    </revision>
    <revision>
      <id>2408888</id>
      <parentid>2408887</parentid>
      <timestamp>2026-05-04T07:44:56Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>dcausse: T425301: stopping writes on cloudelastic</comment>
      <origin>2408888</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="54964" sha1="ecz90he4hfmofkf3wk5m5o65zcv5bdd" xml:space="preserve">
== 2026-05-04 ==
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ecz90he4hfmofkf3wk5m5o65zcv5bdd</sha1>
    </revision>
    <revision>
      <id>2408889</id>
      <parentid>2408888</parentid>
      <timestamp>2026-05-04T07:46:58Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie</comment>
      <origin>2408889</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="55115" sha1="6zccadt8spzggp1e04mes7apkxh7gqz" xml:space="preserve">
== 2026-05-04 ==
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>6zccadt8spzggp1e04mes7apkxh7gqz</sha1>
    </revision>
    <revision>
      <id>2408890</id>
      <parentid>2408889</parentid>
      <timestamp>2026-05-04T07:47:03Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie</comment>
      <origin>2408890</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="55212" sha1="0uz318x4csnyk0po73nobf4e2fcxdtp" xml:space="preserve">
== 2026-05-04 ==
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>0uz318x4csnyk0po73nobf4e2fcxdtp</sha1>
    </revision>
    <revision>
      <id>2408891</id>
      <parentid>2408890</parentid>
      <timestamp>2026-05-04T07:47:31Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie</comment>
      <origin>2408891</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="55328" sha1="b22hsrylvvra7wuw7ttv0slyqjjf7uk" xml:space="preserve">
== 2026-05-04 ==
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>b22hsrylvvra7wuw7ttv0slyqjjf7uk</sha1>
    </revision>
    <revision>
      <id>2408892</id>
      <parentid>2408891</parentid>
      <timestamp>2026-05-04T07:48:26Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply</comment>
      <origin>2408892</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="55462" sha1="74vbh48crr4tm0w1arw6k2v3mro61mn" xml:space="preserve">
== 2026-05-04 ==
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>74vbh48crr4tm0w1arw6k2v3mro61mn</sha1>
    </revision>
    <revision>
      <id>2408893</id>
      <parentid>2408892</parentid>
      <timestamp>2026-05-04T07:48:39Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply</comment>
      <origin>2408893</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="55595" sha1="1unh4hbwlql9rwza9tonqbmtl79e5d3" xml:space="preserve">
== 2026-05-04 ==
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>1unh4hbwlql9rwza9tonqbmtl79e5d3</sha1>
    </revision>
    <revision>
      <id>2408894</id>
      <parentid>2408893</parentid>
      <timestamp>2026-05-04T07:51:23Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply</comment>
      <origin>2408894</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="55705" sha1="teap4kbbr6ay15r0j1juvnbhdvg5wc3" xml:space="preserve">
== 2026-05-04 ==
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>teap4kbbr6ay15r0j1juvnbhdvg5wc3</sha1>
    </revision>
    <revision>
      <id>2408895</id>
      <parentid>2408894</parentid>
      <timestamp>2026-05-04T07:51:43Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply</comment>
      <origin>2408895</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="55814" sha1="ldqxgbsejuh0m8bje5vncgrazpa6e0e" xml:space="preserve">
== 2026-05-04 ==
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ldqxgbsejuh0m8bje5vncgrazpa6e0e</sha1>
    </revision>
    <revision>
      <id>2408896</id>
      <parentid>2408895</parentid>
      <timestamp>2026-05-04T07:55:36Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie</comment>
      <origin>2408896</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="55922" sha1="5dpglnqkyiihb3fs02cg7e040mh6bqi" xml:space="preserve">
== 2026-05-04 ==
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>5dpglnqkyiihb3fs02cg7e040mh6bqi</sha1>
    </revision>
    <revision>
      <id>2408897</id>
      <parentid>2408896</parentid>
      <timestamp>2026-05-04T07:55:54Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet</comment>
      <origin>2408897</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="56023" sha1="62cmlpvk0jfugt3avdcsyp6yghguqa3" xml:space="preserve">
== 2026-05-04 ==
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>62cmlpvk0jfugt3avdcsyp6yghguqa3</sha1>
    </revision>
    <revision>
      <id>2408898</id>
      <parentid>2408897</parentid>
      <timestamp>2026-05-04T07:56:13Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256|Add sva to wmgExtraLanguageNames (T407106)]]</comment>
      <origin>2408898</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="56156" sha1="lq7wm6g457xl1o7g8beh8cciu4gao5t" xml:space="preserve">
== 2026-05-04 ==
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>lq7wm6g457xl1o7g8beh8cciu4gao5t</sha1>
    </revision>
    <revision>
      <id>2408899</id>
      <parentid>2408898</parentid>
      <timestamp>2026-05-04T07:57:17Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie</comment>
      <origin>2408899</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="56274" sha1="dovuytfgmariohzx9qvwbtzfbd4e4ko" xml:space="preserve">
== 2026-05-04 ==
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>dovuytfgmariohzx9qvwbtzfbd4e4ko</sha1>
    </revision>
    <revision>
      <id>2408900</id>
      <parentid>2408899</parentid>
      <timestamp>2026-05-04T07:57:58Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256|Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.</comment>
      <origin>2408900</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="56509" sha1="aql7pm5tfau5l25xrh1xziafgzhz1w7" xml:space="preserve">
== 2026-05-04 ==
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>aql7pm5tfau5l25xrh1xziafgzhz1w7</sha1>
    </revision>
    <revision>
      <id>2408901</id>
      <parentid>2408900</parentid>
      <timestamp>2026-05-04T07:59:49Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment</comment>
      <origin>2408901</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="56580" sha1="ryvcd8kqz44q4k34hqunm7yhlh9p3uu" xml:space="preserve">
== 2026-05-04 ==
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ryvcd8kqz44q4k34hqunm7yhlh9p3uu</sha1>
    </revision>
    <revision>
      <id>2408902</id>
      <parentid>2408901</parentid>
      <timestamp>2026-05-04T08:01:40Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>moritzm: installing Linux 6.1.170 on bookworm hosts</comment>
      <origin>2408902</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="56640" sha1="7xzm7hdko31duwosz3u0ke9fs8n5k9r" xml:space="preserve">
== 2026-05-04 ==
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>7xzm7hdko31duwosz3u0ke9fs8n5k9r</sha1>
    </revision>
    <revision>
      <id>2408903</id>
      <parentid>2408902</parentid>
      <timestamp>2026-05-04T08:01:45Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet</comment>
      <origin>2408903</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="56760" sha1="fr75tifu4kzmvg9gmpah8r1tvb1738y" xml:space="preserve">
== 2026-05-04 ==
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>fr75tifu4kzmvg9gmpah8r1tvb1738y</sha1>
    </revision>
    <revision>
      <id>2408904</id>
      <parentid>2408903</parentid>
      <timestamp>2026-05-04T08:02:24Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync</comment>
      <origin>2408904</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="56855" sha1="q8chwhmwjxwkddtuw3m0r79s74motf7" xml:space="preserve">
== 2026-05-04 ==
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>q8chwhmwjxwkddtuw3m0r79s74motf7</sha1>
    </revision>
    <revision>
      <id>2408905</id>
      <parentid>2408904</parentid>
      <timestamp>2026-05-04T08:02:32Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync</comment>
      <origin>2408905</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="56949" sha1="hg5wk5ow1cwu4qnqu0nq77eng79g7ce" xml:space="preserve">
== 2026-05-04 ==
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>hg5wk5ow1cwu4qnqu0nq77eng79g7ce</sha1>
    </revision>
    <revision>
      <id>2408906</id>
      <parentid>2408905</parentid>
      <timestamp>2026-05-04T08:02:35Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply</comment>
      <origin>2408906</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="57083" sha1="cwf12659c1en2goe2nj61h49dkz68sb" xml:space="preserve">
== 2026-05-04 ==
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>cwf12659c1en2goe2nj61h49dkz68sb</sha1>
    </revision>
    <revision>
      <id>2408907</id>
      <parentid>2408906</parentid>
      <timestamp>2026-05-04T08:02:46Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply</comment>
      <origin>2408907</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="57216" sha1="7wo028xeer8m5t7qb4v7putulhyowss" xml:space="preserve">
== 2026-05-04 ==
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>7wo028xeer8m5t7qb4v7putulhyowss</sha1>
    </revision>
    <revision>
      <id>2408908</id>
      <parentid>2408907</parentid>
      <timestamp>2026-05-04T08:03:38Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync</comment>
      <origin>2408908</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="57309" sha1="le7myhh15iywn4zgq25sgdbfn1ial8l" xml:space="preserve">
== 2026-05-04 ==
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>le7myhh15iywn4zgq25sgdbfn1ial8l</sha1>
    </revision>
    <revision>
      <id>2408909</id>
      <parentid>2408908</parentid>
      <timestamp>2026-05-04T08:04:12Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256|Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)</comment>
      <origin>2408909</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="57463" sha1="pym3qu63warsl5pa22befafotjf0mdi" xml:space="preserve">
== 2026-05-04 ==
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>pym3qu63warsl5pa22befafotjf0mdi</sha1>
    </revision>
    <revision>
      <id>2408910</id>
      <parentid>2408909</parentid>
      <timestamp>2026-05-04T08:04:18Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync</comment>
      <origin>2408910</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="57555" sha1="sw4jx0uwi64r6aewmdf9hot40eanknf" xml:space="preserve">
== 2026-05-04 ==
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>sw4jx0uwi64r6aewmdf9hot40eanknf</sha1>
    </revision>
    <revision>
      <id>2408911</id>
      <parentid>2408910</parentid>
      <timestamp>2026-05-04T08:06:29Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply</comment>
      <origin>2408911</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="57694" sha1="gwfn7xn6rymzvm6kdk4uwk9styindml" xml:space="preserve">
== 2026-05-04 ==
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>gwfn7xn6rymzvm6kdk4uwk9styindml</sha1>
    </revision>
    <revision>
      <id>2408912</id>
      <parentid>2408911</parentid>
      <timestamp>2026-05-04T08:06:49Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply</comment>
      <origin>2408912</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="57832" sha1="33bzkgbmii80jpnbfowj2gic011ug89" xml:space="preserve">
== 2026-05-04 ==
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>33bzkgbmii80jpnbfowj2gic011ug89</sha1>
    </revision>
    <revision>
      <id>2408913</id>
      <parentid>2408912</parentid>
      <timestamp>2026-05-04T08:11:33Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage</comment>
      <origin>2408913</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="57958" sha1="4yxd2dmbsxt9vbx3htxgkjyztdmk515" xml:space="preserve">
== 2026-05-04 ==
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>4yxd2dmbsxt9vbx3htxgkjyztdmk515</sha1>
    </revision>
    <revision>
      <id>2408914</id>
      <parentid>2408913</parentid>
      <timestamp>2026-05-04T08:15:29Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage</comment>
      <origin>2408914</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="58103" sha1="m25uvges3q904n3ylu8i8ly4pw0kaqd" xml:space="preserve">
== 2026-05-04 ==
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>m25uvges3q904n3ylu8i8ly4pw0kaqd</sha1>
    </revision>
    <revision>
      <id>2408915</id>
      <parentid>2408914</parentid>
      <timestamp>2026-05-04T08:19:56Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet</comment>
      <origin>2408915</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="58200" sha1="l9wwc347v8un2hr4lqzlirjsv0sjkew" xml:space="preserve">
== 2026-05-04 ==
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>l9wwc347v8un2hr4lqzlirjsv0sjkew</sha1>
    </revision>
    <revision>
      <id>2408916</id>
      <parentid>2408915</parentid>
      <timestamp>2026-05-04T08:20:12Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance</comment>
      <origin>2408916</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="58350" sha1="e3suksuqkc9or5b3mevvkv12x375zur" xml:space="preserve">
== 2026-05-04 ==
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>e3suksuqkc9or5b3mevvkv12x375zur</sha1>
    </revision>
    <revision>
      <id>2408917</id>
      <parentid>2408916</parentid>
      <timestamp>2026-05-04T08:20:19Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance</comment>
      <origin>2408917</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="58562" sha1="ckofs9odadm91xvbyb61sx034rnnt6q" xml:space="preserve">
== 2026-05-04 ==
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ckofs9odadm91xvbyb61sx034rnnt6q</sha1>
    </revision>
    <revision>
      <id>2408918</id>
      <parentid>2408917</parentid>
      <timestamp>2026-05-04T08:20:25Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 (T419961)', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json</comment>
      <origin>2408918</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="58803" sha1="i6aw8c353cvm0zg8as1pj92alo5yams" xml:space="preserve">
== 2026-05-04 ==
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>i6aw8c353cvm0zg8as1pj92alo5yams</sha1>
    </revision>
    <revision>
      <id>2408920</id>
      <parentid>2408918</parentid>
      <timestamp>2026-05-04T08:23:56Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet</comment>
      <origin>2408920</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="58919" sha1="ri4uomnvs1pnzq6qbzdgj8o4b9akb72" xml:space="preserve">
== 2026-05-04 ==
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ri4uomnvs1pnzq6qbzdgj8o4b9akb72</sha1>
    </revision>
    <revision>
      <id>2408921</id>
      <parentid>2408920</parentid>
      <timestamp>2026-05-04T08:28:50Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 (T419961)', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json</comment>
      <origin>2408921</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="59178" sha1="an026ubyo91i2ahsqhvhcgm6qhgbjoo" xml:space="preserve">
== 2026-05-04 ==
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>an026ubyo91i2ahsqhvhcgm6qhgbjoo</sha1>
    </revision>
    <revision>
      <id>2408922</id>
      <parentid>2408921</parentid>
      <timestamp>2026-05-04T08:32:02Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>moritzm: installing Linux 5.10.251-3 on bullseye hosts</comment>
      <origin>2408922</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="59241" sha1="q9qzhxnvypy83ulkwpskc2qyewz8agx" xml:space="preserve">
== 2026-05-04 ==
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>q9qzhxnvypy83ulkwpskc2qyewz8agx</sha1>
    </revision>
    <revision>
      <id>2408923</id>
      <parentid>2408922</parentid>
      <timestamp>2026-05-04T08:37:43Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie</comment>
      <origin>2408923</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="59368" sha1="eumkm6zv7g49tzgqv4gr5s7gp6rhmom" xml:space="preserve">
== 2026-05-04 ==
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>eumkm6zv7g49tzgqv4gr5s7gp6rhmom</sha1>
    </revision>
    <revision>
      <id>2408924</id>
      <parentid>2408923</parentid>
      <timestamp>2026-05-04T08:38:58Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json</comment>
      <origin>2408924</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="59600" sha1="1e2nb7xew6y00w8sht0v1iufncqlqaj" xml:space="preserve">
== 2026-05-04 ==
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>1e2nb7xew6y00w8sht0v1iufncqlqaj</sha1>
    </revision>
    <revision>
      <id>2408925</id>
      <parentid>2408924</parentid>
      <timestamp>2026-05-04T08:42:26Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet</comment>
      <origin>2408925</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="59732" sha1="j2pxifcpjvfm2gvko0okneozbas2c3b" xml:space="preserve">
== 2026-05-04 ==
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>j2pxifcpjvfm2gvko0okneozbas2c3b</sha1>
    </revision>
    <revision>
      <id>2408926</id>
      <parentid>2408925</parentid>
      <timestamp>2026-05-04T08:42:32Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet</comment>
      <origin>2408926</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="59864" sha1="mk9ak24uq6euk549h96thf46p5jc3d6" xml:space="preserve">
== 2026-05-04 ==
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>mk9ak24uq6euk549h96thf46p5jc3d6</sha1>
    </revision>
    <revision>
      <id>2408927</id>
      <parentid>2408926</parentid>
      <timestamp>2026-05-04T08:42:43Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet</comment>
      <origin>2408927</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="59998" sha1="1ym02extejre5u4x3o46r58bs0dr14r" xml:space="preserve">
== 2026-05-04 ==
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>1ym02extejre5u4x3o46r58bs0dr14r</sha1>
    </revision>
    <revision>
      <id>2408928</id>
      <parentid>2408927</parentid>
      <timestamp>2026-05-04T08:42:49Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet</comment>
      <origin>2408928</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="60132" sha1="0qroltg4qfkmezrkce9my5xpi3jmtes" xml:space="preserve">
== 2026-05-04 ==
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>0qroltg4qfkmezrkce9my5xpi3jmtes</sha1>
    </revision>
    <revision>
      <id>2408929</id>
      <parentid>2408928</parentid>
      <timestamp>2026-05-04T08:42:54Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet</comment>
      <origin>2408929</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="60266" sha1="7ktqa2tjyrtvpqs832q8nsnojww721j" xml:space="preserve">
== 2026-05-04 ==
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>7ktqa2tjyrtvpqs832q8nsnojww721j</sha1>
    </revision>
    <revision>
      <id>2408930</id>
      <parentid>2408929</parentid>
      <timestamp>2026-05-04T08:43:00Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet</comment>
      <origin>2408930</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="60400" sha1="blfq27bs3tk0txdmy08ce74xaa2ccu1" xml:space="preserve">
== 2026-05-04 ==
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>blfq27bs3tk0txdmy08ce74xaa2ccu1</sha1>
    </revision>
    <revision>
      <id>2408931</id>
      <parentid>2408930</parentid>
      <timestamp>2026-05-04T08:43:07Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet</comment>
      <origin>2408931</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="60534" sha1="bqvljpc6in5aaso7vrl0hu7n3b7o563" xml:space="preserve">
== 2026-05-04 ==
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>bqvljpc6in5aaso7vrl0hu7n3b7o563</sha1>
    </revision>
    <revision>
      <id>2408932</id>
      <parentid>2408931</parentid>
      <timestamp>2026-05-04T08:43:12Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet</comment>
      <origin>2408932</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="60668" sha1="dhzslk42t6b2bmrfkly2vm2qakt3tlu" xml:space="preserve">
== 2026-05-04 ==
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>dhzslk42t6b2bmrfkly2vm2qakt3tlu</sha1>
    </revision>
    <revision>
      <id>2408933</id>
      <parentid>2408932</parentid>
      <timestamp>2026-05-04T08:43:20Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet</comment>
      <origin>2408933</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="60802" sha1="7hrsxdc4ad1guvx7sw98tinvdt2l8e5" xml:space="preserve">
== 2026-05-04 ==
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>7hrsxdc4ad1guvx7sw98tinvdt2l8e5</sha1>
    </revision>
    <revision>
      <id>2408934</id>
      <parentid>2408933</parentid>
      <timestamp>2026-05-04T08:43:24Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet</comment>
      <origin>2408934</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="60936" sha1="b2fh3qspu5ise06h181m2guqov4p6te" xml:space="preserve">
== 2026-05-04 ==
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>b2fh3qspu5ise06h181m2guqov4p6te</sha1>
    </revision>
    <revision>
      <id>2408935</id>
      <parentid>2408934</parentid>
      <timestamp>2026-05-04T08:49:05Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json</comment>
      <origin>2408935</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="61168" sha1="rh8w3o6vddungtvf4rmw7l4wgme2kq0" xml:space="preserve">
== 2026-05-04 ==
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>rh8w3o6vddungtvf4rmw7l4wgme2kq0</sha1>
    </revision>
    <revision>
      <id>2408936</id>
      <parentid>2408935</parentid>
      <timestamp>2026-05-04T08:50:45Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie</comment>
      <origin>2408936</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="61267" sha1="ry187i6hh2a9h6w91hve6bmo8nvhu85" xml:space="preserve">
== 2026-05-04 ==
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ry187i6hh2a9h6w91hve6bmo8nvhu85</sha1>
    </revision>
    <revision>
      <id>2408937</id>
      <parentid>2408936</parentid>
      <timestamp>2026-05-04T08:55:52Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .</comment>
      <origin>2408937</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="61390" sha1="85xk3gdaoa6hgiep0vg9exju5pxpb8f" xml:space="preserve">
== 2026-05-04 ==
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>85xk3gdaoa6hgiep0vg9exju5pxpb8f</sha1>
    </revision>
    <revision>
      <id>2408938</id>
      <parentid>2408937</parentid>
      <timestamp>2026-05-04T08:56:05Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .</comment>
      <origin>2408938</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="61513" sha1="2asw3lw9t45sywunyttguyc0abki0yn" xml:space="preserve">
== 2026-05-04 ==
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>2asw3lw9t45sywunyttguyc0abki0yn</sha1>
    </revision>
    <revision>
      <id>2408939</id>
      <parentid>2408938</parentid>
      <timestamp>2026-05-04T08:59:13Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 (T419961)', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json</comment>
      <origin>2408939</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="61772" sha1="tplcafa2jssdboc55gey6vx6aba4s9n" xml:space="preserve">
== 2026-05-04 ==
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>tplcafa2jssdboc55gey6vx6aba4s9n</sha1>
    </revision>
    <revision>
      <id>2408940</id>
      <parentid>2408939</parentid>
      <timestamp>2026-05-04T08:59:23Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance</comment>
      <origin>2408940</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="61922" sha1="96d3wnrj15iso1yrsccjksi577kfgxx" xml:space="preserve">
== 2026-05-04 ==
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>96d3wnrj15iso1yrsccjksi577kfgxx</sha1>
    </revision>
    <revision>
      <id>2408941</id>
      <parentid>2408940</parentid>
      <timestamp>2026-05-04T08:59:32Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 (T419961)', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json</comment>
      <origin>2408941</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="62163" sha1="606nlflij04swjbhq629xmtso698xcj" xml:space="preserve">
== 2026-05-04 ==
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>606nlflij04swjbhq629xmtso698xcj</sha1>
    </revision>
    <revision>
      <id>2408942</id>
      <parentid>2408941</parentid>
      <timestamp>2026-05-04T09:08:46Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 (T419961)', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json</comment>
      <origin>2408942</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="62422" sha1="rkhpvfibru10u93ewtk18dk9485qaci" xml:space="preserve">
== 2026-05-04 ==
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>rkhpvfibru10u93ewtk18dk9485qaci</sha1>
    </revision>
    <revision>
      <id>2408943</id>
      <parentid>2408942</parentid>
      <timestamp>2026-05-04T09:15:11Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events</comment>
      <origin>2408943</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="62571" sha1="gzta95yvihlg4ic7zs6032eqhw2glj9" xml:space="preserve">
== 2026-05-04 ==
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>gzta95yvihlg4ic7zs6032eqhw2glj9</sha1>
    </revision>
    <revision>
      <id>2408944</id>
      <parentid>2408943</parentid>
      <timestamp>2026-05-04T09:15:50Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events</comment>
      <origin>2408944</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="62664" sha1="ly0s1gyx7xrrfgcfg7nxstg4cb7xqwf" xml:space="preserve">
== 2026-05-04 ==
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ly0s1gyx7xrrfgcfg7nxstg4cb7xqwf</sha1>
    </revision>
    <revision>
      <id>2408945</id>
      <parentid>2408944</parentid>
      <timestamp>2026-05-04T09:16:27Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events</comment>
      <origin>2408945</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="62776" sha1="cg7bse3qiof9gk31970t4iby8e0fp09" xml:space="preserve">
== 2026-05-04 ==
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>cg7bse3qiof9gk31970t4iby8e0fp09</sha1>
    </revision>
    <revision>
      <id>2408946</id>
      <parentid>2408945</parentid>
      <timestamp>2026-05-04T09:18:55Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json</comment>
      <origin>2408946</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="63008" sha1="1xe1y6xkjusjuhxl0vtxv4xkt33m47f" xml:space="preserve">
== 2026-05-04 ==
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>1xe1y6xkjusjuhxl0vtxv4xkt33m47f</sha1>
    </revision>
    <revision>
      <id>2408947</id>
      <parentid>2408946</parentid>
      <timestamp>2026-05-04T09:29:03Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json</comment>
      <origin>2408947</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="63240" sha1="jkfbng3ivi53kywodbq1ng90nbapyhb" xml:space="preserve">
== 2026-05-04 ==
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>jkfbng3ivi53kywodbq1ng90nbapyhb</sha1>
    </revision>
    <revision>
      <id>2408948</id>
      <parentid>2408947</parentid>
      <timestamp>2026-05-04T09:36:08Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie</comment>
      <origin>2408948</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="63358" sha1="5bvg9f4g0wgkw01jgsgssjlr3orumri" xml:space="preserve">
== 2026-05-04 ==
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>5bvg9f4g0wgkw01jgsgssjlr3orumri</sha1>
    </revision>
    <revision>
      <id>2408949</id>
      <parentid>2408948</parentid>
      <timestamp>2026-05-04T09:37:15Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply</comment>
      <origin>2408949</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="63499" sha1="qytt8qw6h11eof2lyua4gqnoencduuz" xml:space="preserve">
== 2026-05-04 ==
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>qytt8qw6h11eof2lyua4gqnoencduuz</sha1>
    </revision>
    <revision>
      <id>2408950</id>
      <parentid>2408949</parentid>
      <timestamp>2026-05-04T09:37:17Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply</comment>
      <origin>2408950</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="63639" sha1="ghor87o8chh22kszx7cjc7bi9fducmi" xml:space="preserve">
== 2026-05-04 ==
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ghor87o8chh22kszx7cjc7bi9fducmi</sha1>
    </revision>
    <revision>
      <id>2408951</id>
      <parentid>2408950</parentid>
      <timestamp>2026-05-04T09:37:42Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply</comment>
      <origin>2408951</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="63780" sha1="gjazfmm9g0j2s0bayovkallg149vtri" xml:space="preserve">
== 2026-05-04 ==
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>gjazfmm9g0j2s0bayovkallg149vtri</sha1>
    </revision>
    <revision>
      <id>2408952</id>
      <parentid>2408951</parentid>
      <timestamp>2026-05-04T09:37:54Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply</comment>
      <origin>2408952</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="63920" sha1="6ah4vrs6yquvd01pm15greh5a24ka1e" xml:space="preserve">
== 2026-05-04 ==
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>6ah4vrs6yquvd01pm15greh5a24ka1e</sha1>
    </revision>
    <revision>
      <id>2408953</id>
      <parentid>2408952</parentid>
      <timestamp>2026-05-04T09:39:11Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 (T419961)', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json</comment>
      <origin>2408953</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="64179" sha1="bl61gy32ak3gejb74o95hgfmcvqawpp" xml:space="preserve">
== 2026-05-04 ==
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>bl61gy32ak3gejb74o95hgfmcvqawpp</sha1>
    </revision>
    <revision>
      <id>2408954</id>
      <parentid>2408953</parentid>
      <timestamp>2026-05-04T09:39:30Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance</comment>
      <origin>2408954</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="64329" sha1="muff49fhf8y2oczk1u86h3qncuo6gug" xml:space="preserve">
== 2026-05-04 ==
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>muff49fhf8y2oczk1u86h3qncuo6gug</sha1>
    </revision>
    <revision>
      <id>2408955</id>
      <parentid>2408954</parentid>
      <timestamp>2026-05-04T09:39:39Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 (T419961)', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json</comment>
      <origin>2408955</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="64570" sha1="ajc9g4syo5t25uu8kz49up0n18zn63o" xml:space="preserve">
== 2026-05-04 ==
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ajc9g4syo5t25uu8kz49up0n18zn63o</sha1>
    </revision>
    <revision>
      <id>2408956</id>
      <parentid>2408955</parentid>
      <timestamp>2026-05-04T09:43:24Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org</comment>
      <origin>2408956</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="64666" sha1="jqm4thiku0lkn9xt2gz9spu5b11nbss" xml:space="preserve">
== 2026-05-04 ==
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>jqm4thiku0lkn9xt2gz9spu5b11nbss</sha1>
    </revision>
    <revision>
      <id>2408957</id>
      <parentid>2408956</parentid>
      <timestamp>2026-05-04T09:48:03Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 (T419961)', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json</comment>
      <origin>2408957</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="64925" sha1="n53bbabqogzsqfpq8ootzgu7w7z7cpb" xml:space="preserve">
== 2026-05-04 ==
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>n53bbabqogzsqfpq8ootzgu7w7z7cpb</sha1>
    </revision>
    <revision>
      <id>2408958</id>
      <parentid>2408957</parentid>
      <timestamp>2026-05-04T09:49:43Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org</comment>
      <origin>2408958</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="65040" sha1="ci3mf4h2bb14jfsufqh5de82vn10yqu" xml:space="preserve">
== 2026-05-04 ==
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>ci3mf4h2bb14jfsufqh5de82vn10yqu</sha1>
    </revision>
    <revision>
      <id>2408959</id>
      <parentid>2408958</parentid>
      <timestamp>2026-05-04T09:58:11Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json</comment>
      <origin>2408959</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="65272" sha1="3p0blq6q33zneg3htv10zsbir6kupd5" xml:space="preserve">
== 2026-05-04 ==
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>3p0blq6q33zneg3htv10zsbir6kupd5</sha1>
    </revision>
    <revision>
      <id>2408960</id>
      <parentid>2408959</parentid>
      <timestamp>2026-05-04T10:01:37Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie</comment>
      <origin>2408960</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="65423" sha1="e10mhmve9gz72rcw3k1s5yn1n1opixo" xml:space="preserve">
== 2026-05-04 ==
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>e10mhmve9gz72rcw3k1s5yn1n1opixo</sha1>
    </revision>
    <revision>
      <id>2408961</id>
      <parentid>2408960</parentid>
      <timestamp>2026-05-04T10:01:42Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie</comment>
      <origin>2408961</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="65520" sha1="gxjz0ziniv0in1jn24nzkld3zu8t5gn" xml:space="preserve">
== 2026-05-04 ==
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>gxjz0ziniv0in1jn24nzkld3zu8t5gn</sha1>
    </revision>
    <revision>
      <id>2408962</id>
      <parentid>2408961</parentid>
      <timestamp>2026-05-04T10:02:00Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie</comment>
      <origin>2408962</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="65636" sha1="hs07lrt5x5cgldre6xi7e1q8v18c7l7" xml:space="preserve">
== 2026-05-04 ==
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>hs07lrt5x5cgldre6xi7e1q8v18c7l7</sha1>
    </revision>
    <revision>
      <id>2408963</id>
      <parentid>2408962</parentid>
      <timestamp>2026-05-04T10:02:57Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie</comment>
      <origin>2408963</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="65744" sha1="qoip2uy52pdu2a91decynkpwx65ls26" xml:space="preserve">
== 2026-05-04 ==
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>qoip2uy52pdu2a91decynkpwx65ls26</sha1>
    </revision>
    <revision>
      <id>2408964</id>
      <parentid>2408963</parentid>
      <timestamp>2026-05-04T10:08:19Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json</comment>
      <origin>2408964</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="65976" sha1="jge8j5jtgu043ieowkoq7e2fqygde52" xml:space="preserve">
== 2026-05-04 ==
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>jge8j5jtgu043ieowkoq7e2fqygde52</sha1>
    </revision>
    <revision>
      <id>2408965</id>
      <parentid>2408964</parentid>
      <timestamp>2026-05-04T10:15:50Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage</comment>
      <origin>2408965</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="66102" sha1="dvokktcd9hamz8pgeul4jubs0q35xrn" xml:space="preserve">
== 2026-05-04 ==
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>dvokktcd9hamz8pgeul4jubs0q35xrn</sha1>
    </revision>
    <revision>
      <id>2408966</id>
      <parentid>2408965</parentid>
      <timestamp>2026-05-04T10:16:10Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage</comment>
      <origin>2408966</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="66248" sha1="j3di2qtjpd6o5xooe0i811kugt5weje" xml:space="preserve">
== 2026-05-04 ==
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>j3di2qtjpd6o5xooe0i811kugt5weje</sha1>
    </revision>
    <revision>
      <id>2408967</id>
      <parentid>2408966</parentid>
      <timestamp>2026-05-04T10:16:30Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance</comment>
      <origin>2408967</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="66348" sha1="0r9cui8s265nn1tf9bix8cxuvyn06vd" xml:space="preserve">
== 2026-05-04 ==
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>0r9cui8s265nn1tf9bix8cxuvyn06vd</sha1>
    </revision>
    <revision>
      <id>2408968</id>
      <parentid>2408967</parentid>
      <timestamp>2026-05-04T10:18:27Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 (T419961)', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json</comment>
      <origin>2408968</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="66607" sha1="pt35qqwbnewxjd5w5h3mdumfe9ppcfi" xml:space="preserve">
== 2026-05-04 ==
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>pt35qqwbnewxjd5w5h3mdumfe9ppcfi</sha1>
    </revision>
    <revision>
      <id>2408969</id>
      <parentid>2408968</parentid>
      <timestamp>2026-05-04T10:18:48Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance</comment>
      <origin>2408969</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="66757" sha1="cz1xrg9vkqslc9hxixd6fhnhlq10igb" xml:space="preserve">
== 2026-05-04 ==
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>cz1xrg9vkqslc9hxixd6fhnhlq10igb</sha1>
    </revision>
    <revision>
      <id>2408970</id>
      <parentid>2408969</parentid>
      <timestamp>2026-05-04T10:18:56Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 (T419961)', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json</comment>
      <origin>2408970</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="66998" sha1="s9z4gz3kav3mbi57e0aw3192zzvzygk" xml:space="preserve">
== 2026-05-04 ==
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>s9z4gz3kav3mbi57e0aw3192zzvzygk</sha1>
    </revision>
    <revision>
      <id>2408971</id>
      <parentid>2408970</parentid>
      <timestamp>2026-05-04T10:26:48Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet</comment>
      <origin>2408971</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="67091" sha1="eb9abzrt79yubte2pahuwvn8r7fciq5" xml:space="preserve">
== 2026-05-04 ==
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>eb9abzrt79yubte2pahuwvn8r7fciq5</sha1>
    </revision>
    <revision>
      <id>2408972</id>
      <parentid>2408971</parentid>
      <timestamp>2026-05-04T10:26:49Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.dns.netbox</comment>
      <origin>2408972</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="67146" sha1="23zt2vnfi7w6bhftrp036945acl7u1z" xml:space="preserve">
== 2026-05-04 ==
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>23zt2vnfi7w6bhftrp036945acl7u1z</sha1>
    </revision>
    <revision>
      <id>2408973</id>
      <parentid>2408972</parentid>
      <timestamp>2026-05-04T10:27:16Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 (T419961)', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json</comment>
      <origin>2408973</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="67405" sha1="5cqgczb44w76axjltpntbfwvk257q61" xml:space="preserve">
== 2026-05-04 ==
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>5cqgczb44w76axjltpntbfwvk257q61</sha1>
    </revision>
    <revision>
      <id>2408974</id>
      <parentid>2408973</parentid>
      <timestamp>2026-05-04T10:34:30Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;</comment>
      <origin>2408974</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="67600" sha1="slhobw89yelmb20f0ayae2is7ta8c59" xml:space="preserve">
== 2026-05-04 ==
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>slhobw89yelmb20f0ayae2is7ta8c59</sha1>
    </revision>
    <revision>
      <id>2408975</id>
      <parentid>2408974</parentid>
      <timestamp>2026-05-04T10:34:35Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;</comment>
      <origin>2408975</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="67814" sha1="3drnwdh2x3j6o521nkzax20qa4boxi1" xml:space="preserve">
== 2026-05-04 ==
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>3drnwdh2x3j6o521nkzax20qa4boxi1</sha1>
    </revision>
    <revision>
      <id>2408976</id>
      <parentid>2408975</parentid>
      <timestamp>2026-05-04T10:34:35Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)</comment>
      <origin>2408976</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="67888" sha1="q402eo5hiuljwi7a0l1h8w1kezoqijj" xml:space="preserve">
== 2026-05-04 ==
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>q402eo5hiuljwi7a0l1h8w1kezoqijj</sha1>
    </revision>
    <revision>
      <id>2408977</id>
      <parentid>2408976</parentid>
      <timestamp>2026-05-04T10:34:36Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors</comment>
      <origin>2408977</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="67986" sha1="a3d9tiokajt37zr89qry5bmc34tmczh" xml:space="preserve">
== 2026-05-04 ==
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>a3d9tiokajt37zr89qry5bmc34tmczh</sha1>
    </revision>
    <revision>
      <id>2408978</id>
      <parentid>2408977</parentid>
      <timestamp>2026-05-04T10:34:39Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors</comment>
      <origin>2408978</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="68103" sha1="kc3qzw2nfzuvotmemqrzi2q42cvkw79" xml:space="preserve">
== 2026-05-04 ==
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>kc3qzw2nfzuvotmemqrzi2q42cvkw79</sha1>
    </revision>
    <revision>
      <id>2408979</id>
      <parentid>2408978</parentid>
      <timestamp>2026-05-04T10:35:15Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;</comment>
      <origin>2408979</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="68297" sha1="295x3uwgw8aajxwkgqmdhl89o70svub" xml:space="preserve">
== 2026-05-04 ==
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>295x3uwgw8aajxwkgqmdhl89o70svub</sha1>
    </revision>
    <revision>
      <id>2408980</id>
      <parentid>2408979</parentid>
      <timestamp>2026-05-04T10:35:21Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;</comment>
      <origin>2408980</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="68510" sha1="imq3oz1itgrb1ha4zm05bbkidl8a1v5" xml:space="preserve">
== 2026-05-04 ==
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>imq3oz1itgrb1ha4zm05bbkidl8a1v5</sha1>
    </revision>
    <revision>
      <id>2408981</id>
      <parentid>2408980</parentid>
      <timestamp>2026-05-04T10:37:24Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json</comment>
      <origin>2408981</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="68742" sha1="g9v1acmlteu7odriia029gm3wtk12dj" xml:space="preserve">
== 2026-05-04 ==
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>g9v1acmlteu7odriia029gm3wtk12dj</sha1>
    </revision>
    <revision>
      <id>2408982</id>
      <parentid>2408981</parentid>
      <timestamp>2026-05-04T10:38:38Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm</comment>
      <origin>2408982</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="68848" sha1="golt708m4kaebf3yuobsejvlho10ge8" xml:space="preserve">
== 2026-05-04 ==
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>golt708m4kaebf3yuobsejvlho10ge8</sha1>
    </revision>
    <revision>
      <id>2408983</id>
      <parentid>2408982</parentid>
      <timestamp>2026-05-04T10:39:01Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie</comment>
      <origin>2408983</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="68975" sha1="09u4727qfci0m1rxh7ig3fiulo4tb67" xml:space="preserve">
== 2026-05-04 ==
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>09u4727qfci0m1rxh7ig3fiulo4tb67</sha1>
    </revision>
    <revision>
      <id>2408985</id>
      <parentid>2408983</parentid>
      <timestamp>2026-05-04T10:42:02Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie</comment>
      <origin>2408985</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="69074" sha1="fmz6l0jhtpuvu7u155hcmu4bc0xnouo" xml:space="preserve">
== 2026-05-04 ==
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>fmz6l0jhtpuvu7u155hcmu4bc0xnouo</sha1>
    </revision>
    <revision>
      <id>2408986</id>
      <parentid>2408985</parentid>
      <timestamp>2026-05-04T10:42:14Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>moritzm: installing postgresql-17 security updates</comment>
      <origin>2408986</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="69133" sha1="n5b623jxz7kkjyhqtlq25petl35j0pp" xml:space="preserve">
== 2026-05-04 ==
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>n5b623jxz7kkjyhqtlq25petl35j0pp</sha1>
    </revision>
    <revision>
      <id>2408987</id>
      <parentid>2408986</parentid>
      <timestamp>2026-05-04T10:47:32Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json</comment>
      <origin>2408987</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="69365" sha1="etx14rdjw7z33aiyozr98plavg7xpug" xml:space="preserve">
== 2026-05-04 ==
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>etx14rdjw7z33aiyozr98plavg7xpug</sha1>
    </revision>
    <revision>
      <id>2408988</id>
      <parentid>2408987</parentid>
      <timestamp>2026-05-04T10:48:09Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>moritzm: installing bash updates from trixie point release</comment>
      <origin>2408988</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="69432" sha1="src3bsgb4t2roaxxiiig0z93qyfgn43" xml:space="preserve">
== 2026-05-04 ==
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>src3bsgb4t2roaxxiiig0z93qyfgn43</sha1>
    </revision>
    <revision>
      <id>2408989</id>
      <parentid>2408988</parentid>
      <timestamp>2026-05-04T10:57:40Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 (T419961)', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json</comment>
      <origin>2408989</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="69691" sha1="hldkk0pnizlohoizdfxdkev6qujh2i7" xml:space="preserve">
== 2026-05-04 ==
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>hldkk0pnizlohoizdfxdkev6qujh2i7</sha1>
    </revision>
    <revision>
      <id>2408990</id>
      <parentid>2408989</parentid>
      <timestamp>2026-05-04T10:58:01Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance</comment>
      <origin>2408990</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="69841" sha1="hxhgwl9xngkd3lhj4le9ngaa5p4aemw" xml:space="preserve">
== 2026-05-04 ==
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>hxhgwl9xngkd3lhj4le9ngaa5p4aemw</sha1>
    </revision>
    <revision>
      <id>2408991</id>
      <parentid>2408990</parentid>
      <timestamp>2026-05-04T10:58:09Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 (T419961)', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json</comment>
      <origin>2408991</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="70082" sha1="m4jee5yd9m9u6ma1fgq16xoz8zelv0f" xml:space="preserve">
== 2026-05-04 ==
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>m4jee5yd9m9u6ma1fgq16xoz8zelv0f</sha1>
    </revision>
    <revision>
      <id>2408992</id>
      <parentid>2408991</parentid>
      <timestamp>2026-05-04T11:01:57Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance</comment>
      <origin>2408992</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="70201" sha1="981v2ffvs9v16mow543g4v9rlmyonf0" xml:space="preserve">
== 2026-05-04 ==
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>981v2ffvs9v16mow543g4v9rlmyonf0</sha1>
    </revision>
    <revision>
      <id>2408993</id>
      <parentid>2408992</parentid>
      <timestamp>2026-05-04T11:05:27Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 (T419961)', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json</comment>
      <origin>2408993</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="70460" sha1="l27br0swigd80lw35dsi49lm4nwyy1n" xml:space="preserve">
== 2026-05-04 ==
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>l27br0swigd80lw35dsi49lm4nwyy1n</sha1>
    </revision>
    <revision>
      <id>2408994</id>
      <parentid>2408993</parentid>
      <timestamp>2026-05-04T11:15:35Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json</comment>
      <origin>2408994</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="70692" sha1="nxlht1aq38ngxta9a69u13iuaoclmf3" xml:space="preserve">
== 2026-05-04 ==
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>nxlht1aq38ngxta9a69u13iuaoclmf3</sha1>
    </revision>
    <revision>
      <id>2408995</id>
      <parentid>2408994</parentid>
      <timestamp>2026-05-04T11:25:43Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json</comment>
      <origin>2408995</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="70924" sha1="7l44p2v1m31355yq1rxrz3pgpex6jk0" xml:space="preserve">
== 2026-05-04 ==
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>7l44p2v1m31355yq1rxrz3pgpex6jk0</sha1>
    </revision>
    <revision>
      <id>2408996</id>
      <parentid>2408995</parentid>
      <timestamp>2026-05-04T11:25:49Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage</comment>
      <origin>2408996</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="71046" sha1="r6k6y5wfl9uepmiq5nkd96x3oruie0p" xml:space="preserve">
== 2026-05-04 ==
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>r6k6y5wfl9uepmiq5nkd96x3oruie0p</sha1>
    </revision>
    <revision>
      <id>2408997</id>
      <parentid>2408996</parentid>
      <timestamp>2026-05-04T11:26:14Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage</comment>
      <origin>2408997</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="71188" sha1="sqmbbpkxm1erazr1db7uvj4ghn61ufl" xml:space="preserve">
== 2026-05-04 ==
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>sqmbbpkxm1erazr1db7uvj4ghn61ufl</sha1>
    </revision>
    <revision>
      <id>2408998</id>
      <parentid>2408997</parentid>
      <timestamp>2026-05-04T11:27:27Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie</comment>
      <origin>2408998</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="71306" sha1="lpdmogmnavs2fsqya0yxqlx7p1dcy6k" xml:space="preserve">
== 2026-05-04 ==
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>lpdmogmnavs2fsqya0yxqlx7p1dcy6k</sha1>
    </revision>
    <revision>
      <id>2408999</id>
      <parentid>2408998</parentid>
      <timestamp>2026-05-04T11:35:52Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 (T419961)', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json</comment>
      <origin>2408999</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="71565" sha1="o0oh8tgrxz0e8nfkaxho2a0nzwl1elg" xml:space="preserve">
== 2026-05-04 ==
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>o0oh8tgrxz0e8nfkaxho2a0nzwl1elg</sha1>
    </revision>
    <revision>
      <id>2409000</id>
      <parentid>2408999</parentid>
      <timestamp>2026-05-04T11:36:12Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance</comment>
      <origin>2409000</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="71715" sha1="cp3z68xjhvro99tivgs84zfgsyltk4o" xml:space="preserve">
== 2026-05-04 ==
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>cp3z68xjhvro99tivgs84zfgsyltk4o</sha1>
    </revision>
    <revision>
      <id>2409001</id>
      <parentid>2409000</parentid>
      <timestamp>2026-05-04T11:36:21Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 (T419961)', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json</comment>
      <origin>2409001</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="71956" sha1="6lykt2fhc6q36ty6m0mlfhuicb2d16x" xml:space="preserve">
== 2026-05-04 ==
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>6lykt2fhc6q36ty6m0mlfhuicb2d16x</sha1>
    </revision>
    <revision>
      <id>2409002</id>
      <parentid>2409001</parentid>
      <timestamp>2026-05-04T11:44:01Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 (T419961)', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json</comment>
      <origin>2409002</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="72215" sha1="bavaz3pu07r84u0oqj1v4s7haentsx2" xml:space="preserve">
== 2026-05-04 ==
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>bavaz3pu07r84u0oqj1v4s7haentsx2</sha1>
    </revision>
    <revision>
      <id>2409003</id>
      <parentid>2409002</parentid>
      <timestamp>2026-05-04T11:45:49Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm</comment>
      <origin>2409003</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="72340" sha1="3r32zmccsz6b2bxream6ddbkvqb4v6s" xml:space="preserve">
== 2026-05-04 ==
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>3r32zmccsz6b2bxream6ddbkvqb4v6s</sha1>
    </revision>
    <revision>
      <id>2409004</id>
      <parentid>2409003</parentid>
      <timestamp>2026-05-04T11:45:50Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet</comment>
      <origin>2409004</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="72452" sha1="oq0tj4n5xoz7f4e48fh1vhj0mx1luat" xml:space="preserve">
== 2026-05-04 ==
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>oq0tj4n5xoz7f4e48fh1vhj0mx1luat</sha1>
    </revision>
    <revision>
      <id>2409005</id>
      <parentid>2409004</parentid>
      <timestamp>2026-05-04T11:47:16Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5004.eqsin.wmnet</comment>
      <origin>2409005</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="72545" sha1="t1qcagw72yccgb00g726gi1q7scay4m" xml:space="preserve">
== 2026-05-04 ==
* 11:47 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5004.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>t1qcagw72yccgb00g726gi1q7scay4m</sha1>
    </revision>
    <revision>
      <id>2409006</id>
      <parentid>2409005</parentid>
      <timestamp>2026-05-04T11:47:18Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.dns.netbox</comment>
      <origin>2409006</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="72600" sha1="mdw58j15dsjecsfhneud4qembc6egzj" xml:space="preserve">
== 2026-05-04 ==
* 11:47 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 11:47 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5004.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>mdw58j15dsjecsfhneud4qembc6egzj</sha1>
    </revision>
    <revision>
      <id>2409007</id>
      <parentid>2409006</parentid>
      <timestamp>2026-05-04T11:51:04Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;</comment>
      <origin>2409007</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="72795" sha1="t1wbytgj66vnxoqltpvumqctwo338sf" xml:space="preserve">
== 2026-05-04 ==
* 11:51 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:47 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 11:47 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5004.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>t1wbytgj66vnxoqltpvumqctwo338sf</sha1>
    </revision>
    <revision>
      <id>2409008</id>
      <parentid>2409007</parentid>
      <timestamp>2026-05-04T11:54:09Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P92197 and previous config saved to /var/cache/conftool/dbconfig/20260504-115408-fceratto.json</comment>
      <origin>2409008</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="73027" sha1="2l6f48rv8mfsf1pvy2b7wiimmk09uza" xml:space="preserve">
== 2026-05-04 ==
* 11:54 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P92197 and previous config saved to /var/cache/conftool/dbconfig/20260504-115408-fceratto.json
* 11:51 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:47 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 11:47 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5004.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>2l6f48rv8mfsf1pvy2b7wiimmk09uza</sha1>
    </revision>
    <revision>
      <id>2409009</id>
      <parentid>2409008</parentid>
      <timestamp>2026-05-04T11:55:13Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;</comment>
      <origin>2409009</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="73241" sha1="s44f7qbylmvkj3xzlwc8556bkfvxtny" xml:space="preserve">
== 2026-05-04 ==
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:54 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P92197 and previous config saved to /var/cache/conftool/dbconfig/20260504-115408-fceratto.json
* 11:51 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:47 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 11:47 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5004.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>s44f7qbylmvkj3xzlwc8556bkfvxtny</sha1>
    </revision>
    <revision>
      <id>2409010</id>
      <parentid>2409009</parentid>
      <timestamp>2026-05-04T11:55:13Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)</comment>
      <origin>2409010</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="73315" sha1="nyzlvfjxx8lmcij9r55ev7hza2c97ry" xml:space="preserve">
== 2026-05-04 ==
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:54 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P92197 and previous config saved to /var/cache/conftool/dbconfig/20260504-115408-fceratto.json
* 11:51 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:47 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 11:47 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5004.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>nyzlvfjxx8lmcij9r55ev7hza2c97ry</sha1>
    </revision>
    <revision>
      <id>2409011</id>
      <parentid>2409010</parentid>
      <timestamp>2026-05-04T11:55:14Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5004.eqsin.wmnet on all recursors</comment>
      <origin>2409011</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="73413" sha1="4cpd7pw5jnd9xgdvxte067t4cq68hxo" xml:space="preserve">
== 2026-05-04 ==
* 11:55 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5004.eqsin.wmnet on all recursors
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:54 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P92197 and previous config saved to /var/cache/conftool/dbconfig/20260504-115408-fceratto.json
* 11:51 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:47 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 11:47 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5004.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>4cpd7pw5jnd9xgdvxte067t4cq68hxo</sha1>
    </revision>
    <revision>
      <id>2409012</id>
      <parentid>2409011</parentid>
      <timestamp>2026-05-04T11:55:17Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5004.eqsin.wmnet on all recursors</comment>
      <origin>2409012</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="73530" sha1="2fj1o5izesmez3w8nk1i7eqp5w1eiix" xml:space="preserve">
== 2026-05-04 ==
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5004.eqsin.wmnet on all recursors
* 11:55 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5004.eqsin.wmnet on all recursors
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:54 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P92197 and previous config saved to /var/cache/conftool/dbconfig/20260504-115408-fceratto.json
* 11:51 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:47 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 11:47 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5004.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>2fj1o5izesmez3w8nk1i7eqp5w1eiix</sha1>
    </revision>
    <revision>
      <id>2409013</id>
      <parentid>2409012</parentid>
      <timestamp>2026-05-04T11:55:52Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;</comment>
      <origin>2409013</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="73724" sha1="f9m365j42vnyl8fjfoqt70dqdec3odd" xml:space="preserve">
== 2026-05-04 ==
* 11:55 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5004.eqsin.wmnet on all recursors
* 11:55 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5004.eqsin.wmnet on all recursors
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:54 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P92197 and previous config saved to /var/cache/conftool/dbconfig/20260504-115408-fceratto.json
* 11:51 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:47 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 11:47 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5004.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>f9m365j42vnyl8fjfoqt70dqdec3odd</sha1>
    </revision>
    <revision>
      <id>2409014</id>
      <parentid>2409013</parentid>
      <timestamp>2026-05-04T11:55:56Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;</comment>
      <origin>2409014</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="73937" sha1="q3fnq3msdx2r3f2ap0de57f8pv4fhxp" xml:space="preserve">
== 2026-05-04 ==
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:55 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5004.eqsin.wmnet on all recursors
* 11:55 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5004.eqsin.wmnet on all recursors
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 11:55 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:54 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P92197 and previous config saved to /var/cache/conftool/dbconfig/20260504-115408-fceratto.json
* 11:51 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5004.eqsin.wmnet - jmm@cumin2002&quot;
* 11:47 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 11:47 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5004.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum5003.eqsin.wmnet
* 11:45 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum5003.eqsin.wmnet with OS bookworm
* 11:44 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92196 and previous config saved to /var/cache/conftool/dbconfig/20260504-114400-fceratto.json
* 11:36 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1193 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92195 and previous config saved to /var/cache/conftool/dbconfig/20260504-113620-fceratto.json
* 11:36 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance
* 11:35 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92194 and previous config saved to /var/cache/conftool/dbconfig/20260504-113550-fceratto.json
* 11:27 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1162: after reimage to trixie
* 11:26 jmm@cumin2002: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 jmm@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on durum5003.eqsin.wmnet with reason: host reimage
* 11:25 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92192 and previous config saved to /var/cache/conftool/dbconfig/20260504-112542-fceratto.json
* 11:15 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P92191 and previous config saved to /var/cache/conftool/dbconfig/20260504-111534-fceratto.json
* 11:05 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92189 and previous config saved to /var/cache/conftool/dbconfig/20260504-110526-fceratto.json
* 11:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2187: repool after maintenance
* 10:58 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1192 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92187 and previous config saved to /var/cache/conftool/dbconfig/20260504-105808-fceratto.json
* 10:58 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance
* 10:57 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92186 and previous config saved to /var/cache/conftool/dbconfig/20260504-105739-fceratto.json
* 10:48 moritzm: installing bash updates from trixie point release
* 10:47 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92184 and previous config saved to /var/cache/conftool/dbconfig/20260504-104731-fceratto.json
* 10:42 moritzm: installing postgresql-17 security updates
* 10:42 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1162: after reimage to trixie
* 10:39 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1162.eqiad.wmnet with OS trixie
* 10:38 jmm@cumin2002: START - Cookbook sre.hosts.reimage for host durum5003.eqsin.wmnet with OS bookworm
* 10:37 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P92181 and previous config saved to /var/cache/conftool/dbconfig/20260504-103723-fceratto.json
* 10:35 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:35 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.ganeti.makevm: created new VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: START - Cookbook sre.dns.wipe-cache durum5003.eqsin.wmnet on all recursors
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 10:34 jmm@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:34 jmm@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Add records for VM durum5003.eqsin.wmnet - jmm@cumin2002&quot;
* 10:27 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92179 and previous config saved to /var/cache/conftool/dbconfig/20260504-102715-fceratto.json
* 10:26 jmm@cumin2002: START - Cookbook sre.dns.netbox
* 10:26 jmm@cumin2002: START - Cookbook sre.ganeti.makevm for new host durum5003.eqsin.wmnet
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1178 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92178 and previous config saved to /var/cache/conftool/dbconfig/20260504-101855-fceratto.json
* 10:18 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance
* 10:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92177 and previous config saved to /var/cache/conftool/dbconfig/20260504-101826-fceratto.json
* 10:16 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2187: repool after maintenance
* 10:16 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:15 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1162.eqiad.wmnet with reason: host reimage
* 10:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92174 and previous config saved to /var/cache/conftool/dbconfig/20260504-100818-fceratto.json
* 10:02 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1162.eqiad.wmnet with OS trixie
* 10:01 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1162: Reimage to Trixie
* 10:01 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1162.eqiad.wmnet with reason: Reimage to Trixie
* 09:58 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P92172 and previous config saved to /var/cache/conftool/dbconfig/20260504-095810-fceratto.json
* 09:49 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org
* 09:48 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92171 and previous config saved to /var/cache/conftool/dbconfig/20260504-094802-fceratto.json
* 09:43 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1177 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92170 and previous config saved to /var/cache/conftool/dbconfig/20260504-093938-fceratto.json
* 09:39 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance
* 09:39 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92169 and previous config saved to /var/cache/conftool/dbconfig/20260504-093910-fceratto.json
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:37 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 09:36 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1182: after reimage to trixie
* 09:29 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92167 and previous config saved to /var/cache/conftool/dbconfig/20260504-092902-fceratto.json
* 09:18 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P92165 and previous config saved to /var/cache/conftool/dbconfig/20260504-091853-fceratto.json
* 09:16 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2187: Fixing events
* 09:15 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2187: Fixing events
* 09:15 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2187.codfw.wmnet with reason: Checking events
* 09:08 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92163 and previous config saved to /var/cache/conftool/dbconfig/20260504-090845-fceratto.json
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1172 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92161 and previous config saved to /var/cache/conftool/dbconfig/20260504-085930-fceratto.json
* 08:59 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance
* 08:59 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92160 and previous config saved to /var/cache/conftool/dbconfig/20260504-085912-fceratto.json
* 08:56 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:55 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 08:50 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1182: after reimage to trixie
* 08:49 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92158 and previous config saved to /var/cache/conftool/dbconfig/20260504-084904-fceratto.json
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1008.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1007.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1006.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1005.eqiad.wmnet
* 08:43 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1004.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1003.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-worker1001.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1002.eqiad.wmnet
* 08:42 jmm@cumin2002: DONE (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: tools-k8s-ctrl1001.eqiad.wmnet
* 08:38 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P92157 and previous config saved to /var/cache/conftool/dbconfig/20260504-083857-fceratto.json
* 08:37 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1182.eqiad.wmnet with OS trixie
* 08:32 moritzm: installing Linux 5.10.251-3 on bullseye hosts
* 08:28 fceratto@cumin1003: dbctl commit (dc=all): 'Repooling after maintenance db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92156 and previous config saved to /var/cache/conftool/dbconfig/20260504-082849-fceratto.json
* 08:23 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host webperf1003.eqiad.wmnet
* 08:20 fceratto@cumin1003: dbctl commit (dc=all): 'Depooling db1167 ([[phab:T419961|T419961]])', diff saved to https://phabricator.wikimedia.org/P92155 and previous config saved to /var/cache/conftool/dbconfig/20260504-082024-fceratto.json
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance
* 08:20 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1167.eqiad.wmnet with reason: Maintenance
* 08:19 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host webperf1003.eqiad.wmnet
* 08:15 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:11 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1182.eqiad.wmnet with reason: host reimage
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:06 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply
* 08:04 gkyziridis@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventstreams: sync
* 08:04 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] (duration: 07m 58s)
* 08:03 gkyziridis@deploy1003: helmfile [eqiad] START helmfile.d/services/eventstreams: sync
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 08:02 gkyziridis@deploy1003: helmfile [staging] DONE helmfile.d/services/eventstreams: sync
* 08:02 gkyziridis@deploy1003: helmfile [staging] START helmfile.d/services/eventstreams: sync
* 08:01 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet
* 08:01 moritzm: installing Linux 6.1.170 on bookworm hosts
* 07:59 urbanecm@deploy1003: urbanecm, h2o: Continuing with deployment
* 07:57 urbanecm@deploy1003: urbanecm, h2o: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 07:57 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1212: after reimage to trixie
* 07:56 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1277256{{!}}Add sva to wmgExtraLanguageNames (T407106)]]
* 07:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet
* 07:55 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1182.eqiad.wmnet with OS trixie
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:51 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:48 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:47 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1182: Reimage to Trixie
* 07:47 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1182: Reimage to Trixie
* 07:46 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1182.eqiad.wmnet with reason: Reimage to Trixie
* 07:44 dcausse: [[phab:T425301|T425301]]: stopping writes on cloudelastic
* 07:44 dcausse@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 07:44 dcausse@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db2147.codfw.wmnet
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 07:42 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:42 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: db2147.codfw.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003&quot;
* 07:41 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2149: after reimage to trixie
* 07:40 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1188: after reimage to trixie
* 07:38 moritzm: installing Linux 6.12.85 on trixie hosts
* 07:37 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-jumbo2003.codfw.wmnet
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:35 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply
* 07:33 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-jumbo2003.codfw.wmnet
* 07:33 marostegui@cumin1003: START - Cookbook sre.dns.netbox
* 07:28 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db2147.codfw.wmnet
* 07:20 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org
* 07:16 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org
* 07:11 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1212: after reimage to trixie
* 07:05 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1212.eqiad.wmnet with OS trixie
* 06:56 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2149: after reimage to trixie
* 06:55 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1188: after reimage to trixie
* 06:52 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1188.eqiad.wmnet with OS trixie
* 06:47 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2149.codfw.wmnet with OS trixie
* 06:43 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:37 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1212.eqiad.wmnet with reason: host reimage
* 06:29 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:25 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:21 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1212.eqiad.wmnet with OS trixie
* 06:19 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1188.eqiad.wmnet with reason: host reimage
* 06:17 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2149.codfw.wmnet with reason: host reimage
* 06:11 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1212: Reimage to Trixie
* 06:11 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1212.eqiad.wmnet with reason: Reimage to Trixie
* 06:10 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 13 hosts with reason: Sanitarium s3 master: reimage to Debian Trixie
* 06:09 marostegui: Reimage sanitarium master for s3, lag to be expected on wikireplicas for s3 [[phab:T424792|T424792]]
* 06:05 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db1188.eqiad.wmnet with OS trixie
* 06:02 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1188: Reimage to Trixie
* 05:58 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1188.eqiad.wmnet with reason: Reimage to Trixie
* 05:57 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host db2149.codfw.wmnet with OS trixie
* 05:55 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2149: Reimage to Trixie
* 05:55 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2149: Reimage to Trixie
* 05:54 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Reimage to Trixie
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 36s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image

== 2026-05-03 ==
* 14:11 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] (duration: 10m 51s)
* 14:05 ladsgroup@deploy1003: ladsgroup: Continuing with deployment
* 14:04 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 14:00 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1281991{{!}}Disable uploads in scnwiki (T425278)]]
* 12:27 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]] (duration: 29m 22s)
* 11:58 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281963{{!}}Remove Wikinews from installer's default main page]]

== 2026-05-02 ==
* 23:32 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] (duration: 06m 41s)
* 23:28 zabe@deploy1003: dreamyjazz, zabe: Continuing with deployment
* 23:27 zabe@deploy1003: dreamyjazz, zabe: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:26 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281775{{!}}Uninstall DynamicPageList from wikis it's not used on (T425202)]]
* 23:22 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] (duration: 07m 27s)
* 23:18 zabe@deploy1003: zabe, dreamyjazz: Continuing with deployment
* 23:17 zabe@deploy1003: zabe, dreamyjazz: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 23:15 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1281739{{!}}Uninstall DynamicPageList from officewiki (T425154)]]
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2014.codfw.wmnet with OS trixie
* 18:07 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host rdb2013.codfw.wmnet with OS trixie
* 18:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:59 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host rdb2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 17:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2369.codfw.wmnet with OS trixie
* 17:42 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:36 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2369.codfw.wmnet with reason: host reimage
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2374.codfw.wmnet with OS trixie
* 17:14 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:13 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2373.codfw.wmnet with OS trixie
* 17:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:09 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2372.codfw.wmnet with OS trixie
* 17:06 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:05 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 17:01 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2370.codfw.wmnet with OS trixie
* 17:00 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:56 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:56 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:52 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:51 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 16:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:44 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2374.codfw.wmnet with reason: host reimage
* 16:43 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2373.codfw.wmnet with reason: host reimage
* 16:42 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2372.codfw.wmnet with reason: host reimage
* 16:36 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:32 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2371.codfw.wmnet with reason: host reimage
* 16:31 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2374.codfw.wmnet with OS trixie
* 16:30 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2373.codfw.wmnet with OS trixie
* 16:29 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2372.codfw.wmnet with OS trixie
* 16:28 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2370.codfw.wmnet with reason: host reimage
* 16:20 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2371.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2370.codfw.wmnet with OS trixie
* 16:16 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2369.codfw.wmnet with OS trixie
* 16:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 16:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 15:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:43 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 15:38 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 15:36 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 15:25 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 15:24 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 15:23 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 12:02 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] (duration: 13m 06s)
* 11:57 samtar@deploy1003: samtar: Continuing with deployment
* 11:50 samtar@deploy1003: samtar: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 11:49 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281747{{!}}Watchlist star: Revert popover/dialog changes (T425185)]]
* 09:20 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-media: apply
* 09:19 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-media: apply
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2366.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2368.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host wikikube-worker2367.codfw.wmnet with OS trixie
* 02:52 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:52 jhancock@cumin2002: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:17 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:12 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 31s)
* 02:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 01:58 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:54 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2368.codfw.wmnet with reason: host reimage
* 01:50 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2367.codfw.wmnet with reason: host reimage
* 01:49 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2366.codfw.wmnet with reason: host reimage
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2368.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2367.codfw.wmnet with OS trixie
* 01:37 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2366.codfw.wmnet with OS trixie
* 01:35 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2364.codfw.wmnet with OS trixie
* 01:34 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:29 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2365.codfw.wmnet with OS trixie
* 01:24 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:23 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2363.codfw.wmnet with OS trixie
* 01:21 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:20 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 01:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 01:06 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 01:03 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:58 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2365.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2364.codfw.wmnet with reason: host reimage
* 00:57 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2363.codfw.wmnet with reason: host reimage
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2365.codfw.wmnet with OS trixie
* 00:45 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2364.codfw.wmnet with OS trixie
* 00:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2363.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2362.codfw.wmnet with OS trixie
* 00:09 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:07 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2361.codfw.wmnet with OS trixie
* 00:05 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:02 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2360.codfw.wmnet with OS trixie
* 00:01 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 00:01 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;

== 2026-05-01 ==
* 23:51 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:47 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:45 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2362.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2361.codfw.wmnet with reason: host reimage
* 23:39 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2360.codfw.wmnet with reason: host reimage
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2362.codfw.wmnet with OS trixie
* 23:27 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2361.codfw.wmnet with OS trixie
* 23:26 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2360.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2357.codfw.wmnet with OS trixie
* 23:25 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:25 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2359.codfw.wmnet with OS trixie
* 23:22 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:22 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2358.codfw.wmnet with OS trixie
* 23:18 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:18 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002&quot;
* 23:08 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 23:05 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 23:02 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2359.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2357.codfw.wmnet with reason: host reimage
* 22:56 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2358.codfw.wmnet with reason: host reimage
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2359.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2358.codfw.wmnet with OS trixie
* 22:44 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-worker2357.codfw.wmnet with OS trixie
* 22:39 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:28 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:23 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:19 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2374.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2373.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2372.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:15 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:06 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 22:02 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:59 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:57 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:51 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:50 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2371.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:49 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2370.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:48 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2369.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:46 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2368.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:41 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2367.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:40 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2366.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:38 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2365.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:33 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2364.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:32 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:31 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2363.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:26 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2362.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:25 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:24 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:21 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:19 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:18 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2361.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:17 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2360.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:15 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:14 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:13 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:10 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:07 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2359.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2358.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:06 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host wikikube-worker2357.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2374
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2374
* 21:04 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2373
* 21:04 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2373
* 20:59 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2372
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2372
* 20:58 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2371
* 20:58 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2371
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2370
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2370
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2369
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2369
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2368
* 20:57 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2368
* 20:57 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2367
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2367
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2366
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2366
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2365
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2365
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2364
* 20:56 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2364
* 20:56 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2363
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2363
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2362
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2362
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2361
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2361
* 20:55 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2360
* 20:55 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2360
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2359
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2359
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2358
* 20:54 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2358
* 20:54 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2357
* 20:53 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2357
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 20:53 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:53 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2357 to codfw - jhancock@cumin2002&quot;
* 20:49 jhancock@cumin2002: START - Cookbook sre.dns.netbox
* 20:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2002.codfw.wmnet with OS trixie
* 20:06 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] (duration: 15m 27s)
* 20:02 krinkle@deploy1003: krinkle: Continuing with deployment
* 19:54 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:52 krinkle@deploy1003: krinkle: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 19:51 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1269440{{!}}Enable wgTrackMediaRequestProvenance on wikidata.org (T414338)]], [[gerrit:1269441{{!}}Enable wgTrackMediaRequestProvenance on Commons (T414338)]]
* 19:49 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2002.codfw.wmnet with reason: host reimage
* 19:40 dancy@deploy1003: Finished scap sync-world: testing [[phab:T317405|T317405]] (duration: 03m 23s)
* 19:37 dancy@deploy1003: Started scap sync-world: testing [[phab:T317405|T317405]]
* 19:36 dancy@deploy1003: Installation of scap version &quot;4.259.0&quot; completed for 2 hosts
* 19:34 dancy@deploy1003: Installing scap version &quot;4.259.0&quot; for 2 host(s)
* 18:55 elukey@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'sync'.
* 18:55 elukey@deploy1003: helmfile [codfw] START helmfile.d/admin 'sync'.
* 18:43 jhathaway@cumin1003: DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Alangi Derick out of all services on: 2442 hosts
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2002
* 18:41 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2002
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2002.codfw.wmnet 50.16.192.10.in-addr.arpa 0.5.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 18:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2002 - herron@cumin1003&quot;
* 18:36 herron@cumin1003: START - Cookbook sre.dns.netbox
* 18:33 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2002
* 18:32 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2002.codfw.wmnet with OS trixie
* 18:26 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2003.codfw.wmnet with OS trixie
* 18:04 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 18:00 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2003.codfw.wmnet with reason: host reimage
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2003
* 17:41 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2003
* 17:40 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2003
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2003.codfw.wmnet 24.32.192.10.in-addr.arpa 4.2.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 17:40 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:40 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2003 - herron@cumin1003&quot;
* 17:33 herron@cumin1003: START - Cookbook sre.dns.netbox
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2003
* 17:28 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2003.codfw.wmnet with OS trixie
* 17:15 herron@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging2004.codfw.wmnet with OS trixie
* 16:34 cdobbins@cumin2002: conftool action : get/pooled; selector: name=cp5024.eqsin.wmnet
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply
* 16:30 ebernhardson@deploy1003: helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply
* 16:08 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet
* 16:02 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet
* 16:00 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2001.codfw.wmnet
* 15:59 aikochou@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' .
* 15:55 jmm@cumin2002: START - Cookbook sre.hosts.reboot-single for host sretest2001.codfw.wmnet
* 15:47 dancy@deploy1003: Installation of scap version &quot;4.258.1&quot; completed for 2 hosts
* 15:45 dancy@deploy1003: Installing scap version &quot;4.258.1&quot; for 2 host(s)
* 15:34 herron@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:30 herron@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging2004.codfw.wmnet with reason: host reimage
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host kafka-logging2004
* 15:14 herron@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host kafka-logging2004
* 15:11 herron@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host kafka-logging2004
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: START - Cookbook sre.dns.wipe-cache kafka-logging2004.codfw.wmnet 38.16.192.10.in-addr.arpa 8.3.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 15:11 herron@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:11 herron@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: Update records for host kafka-logging2004 - herron@cumin1003&quot;
* 15:05 dancy@deploy1003: Installation of scap version &quot;4.258.0&quot; completed for 2 hosts
* 15:03 dancy@deploy1003: Installing scap version &quot;4.258.0&quot; for 2 host(s)
* 14:57 herron@cumin1003: START - Cookbook sre.dns.netbox
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.move-vlan for host kafka-logging2004
* 14:47 herron@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging2004.codfw.wmnet with OS trixie
* 13:45 zabe@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply
* 13:44 zabe@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply
* 13:24 _Gerges: WikiMonitor setup
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1080
* 13:09 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1079
* 13:09 jclark@cumin1003: END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudvirt1077
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1080
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1079
* 13:09 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078
* 13:08 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1077
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:08 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:07 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1077.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:06 jclark@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
* 13:04 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:04 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: &quot;Triggered by cookbooks.sre.dns.netbox: adding cloudvirt1077 to eqiad - jclark@cumin1003&quot;
* 13:00 jclark@cumin1003: START - Cookbook sre.dns.netbox
* 12:34 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply
* 12:34 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply
* 12:33 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply
* 09:57 samtar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] (duration: 06m 49s)
* 09:53 samtar@deploy1003: samtar: Continuing with deployment
* 09:52 samtar@deploy1003: samtar: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 09:50 samtar@deploy1003: Started scap sync-world: Backport for [[gerrit:1281423{{!}}Switch watchstar from Popover to Dialog (T417847)]]
* 09:38 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]] (duration: 06m 05s)
* 09:32 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1281426{{!}}Update the interwiki cache (T239173)]]
* 08:13 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply
* 08:13 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply
* 08:12 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 03:26 akhatun@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich: apply
* 02:07 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 41s)
* 02:00 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image
* 00:16 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] (duration: 07m 05s)
* 00:13 zabe@deploy1003: zabe: Continuing with deployment
* 00:11 zabe@deploy1003: zabe: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
* 00:09 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1280417{{!}}Add script to fix fr_deleted drifts (T424553)]]

== Other archives ==
See [[Server Admin Log/Archives]].

&lt;noinclude&gt;
[[Category:SAL]]
[[Category:Operations]]
&lt;/noinclude&gt;</text>
      <sha1>q3fnq3msdx2r3f2ap0de57f8pv4fhxp</sha1>
    </revision>
  </page>
  <page>
    <title>Release Engineering/SAL</title>
    <ns>0</ns>
    <id>17290</id>
    <revision>
      <id>2408984</id>
      <parentid>2408744</parentid>
      <timestamp>2026-05-04T10:40:58Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>James_F: Zuul: Provide non-voting PHP 8.4/8.5 Quibble jobs for bluespice template</comment>
      <origin>2408984</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="42129" sha1="jjj35r9gxfs24i69qt9q19ma3denzlr" xml:space="preserve">=== 2026-05-04 ===
* 10:40 James_F: Zuul: Provide non-voting PHP 8.4/8.5 Quibble jobs for bluespice template

=== 2026-05-02 ===
* 20:49 James_F: Zuul: [mediawiki/core] Enforce PHP 8.4 &amp; 8.5 on release branches, all pass
* 19:27 James_F: Zuul: Provide non-voting PHP 8.4/8.5 Quibble jobs for MW release branches
* 19:19 James_F: Zuul: [mediawiki/extensions/BlogPage] Add dependencies
* 16:48 James_F: Hard-restarting Zuul to clear the huge number of i18n updates being re-submitted.
* 15:48 James_F: Zuul: [wikimedia-cz/*] Test in PHP 8.3+, dropping 8.2
* 14:02 TheresNoTime: Add bvibber to deployment-prep project
* 09:08 James_F: Docker: [quibble-*] Add php-luasandbox so we can test both modes in Scribunto

=== 2026-05-01 ===
* 15:42 James_F: Zuul: [wikimedia/lucene-explain-parser] Test in PHP 8.3+, dropping 8.2
* 15:42 James_F: Zuul: [wikimedia/textcat] Test in PHP 8.3+, dropping 8.2
* 15:42 James_F: Zuul: [mediawiki/tools/ParseWiki] Test in PHP 8.3+, dropping 8.2
* 15:42 James_F: zuul: Add ToprakM to CI allowlist
* 15:19 James_F: Zuul: [translatewiki] Test in PHP 8.3+, dropping 8.2
* 15:10 James_F: Zuul: [mediawiki/extensions/WikiEditor] Add TestKitchen as a dependency, for [[phab:T425076|T425076]]
* 12:40 James_F: Zuul: [mediawiki/tools/code-utils] Test in PHP 8.3+, dropping 8.2
* 08:02 James_F: Zuul: Update xtex's e-mail in the allowlist
* 07:37 James_F: Zuul: Switch release branches' selenium jobs to PHP 8.3
* 07:33 James_F: Zuul: Test Wikimedia production libraries in PHP 8.3+, dropping 8.2

=== 2026-04-30 ===
* 21:36 brennen: gitlab-webhooks: building &amp; restarting to deploy https://gitlab.wikimedia.org/repos/releng/gitlab-webhooks/-/merge_requests/40
* 20:26 James_F: Zuul: [mediawiki/tools/api-testing] Make PHP 8.5 CI voting
* 20:16 James_F: jforrester@doc1004:~$ # sudo -u doc-uploader rm -rf /srv/doc/cover-extensions/WebAuthn/ # [[phab:T415832|T415832]]
* 20:14 James_F: Zuul: [mediawiki/extensions/WebAuthn] Archive, for [[phab:T415832|T415832]] / [[phab:T303495|T303495]]
* 17:16 brennen: wikibugs: most maintainers at hackathon, so go release-engineering added as a maintainer while looking to debug error at https://gitlab.wikimedia.org/toolforge-repos/wikibugs2/-/jobs/810904
* 15:19 mutante: upgrading zuul to 14.2.0-1 on &quot;new zuul&quot; machines ([[phab:T424879|T424879]])

=== 2026-04-29 ===
* 15:49 James_F: Zuul: [mediawiki/extensions/DiscussionTools] Add ConfirmEdit dependency, for [[phab:T424597|T424597]]
* 15:36 James_F: Zuul: Drop experimental node22 jobs, never used in practice
* 15:28 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1279392, https://gerrit.wikimedia.org/r/1279397

=== 2026-04-28 ===
* 18:11 bd808: Unblock 86.0.0.0/16
* 17:41 bd808: Unblock 79.192.0.0/10
* 17:07 James_F: Zuul: [mediawiki/tools/phpunit-patch-coverage] Drop PHP 8.2 testing
* 17:07 James_F: Zuul: [mediawiki/tools/minus-x] Drop PHP 8.2 testing
* 17:07 James_F: Zuul: [mediawiki/tools/codesniffer] Drop PHP 8.2 testing
* 16:32 James_F: Zuul: [mediawiki/services/jobrunner] Drop PHP 8.2 testing
* 13:34 James_F: Zuul: [mediawiki/tools/phan] Drop PHP 8.2 testing
* 13:34 James_F: Zuul: [oojs/ui] Drop PHP 8.2 testing
* 13:14 James_F: Zuul: [mediawiki/tools/phan/SecurityCheckPlugin] Drop PHP 8.2 CI
* 10:40 Silvan_WMDE: sudo -u jenkins-deploy rm -fR /srv/castor/castor-mw-ext-and-skins/master/mwext-node24-rundoc/ # run on integration-castor06.integration.eqiad1.wikimedia.cloud to fix failure seen in mwext-node24-rundoc #1717
* 00:03 bd808: Increase parallelism for wmf-beta-update-databases.py ([[phab:T256168|T256168]])

=== 2026-04-27 ===
* 22:11 bd808: Beta Cluster MediaWiki update logs now available via https://beta-update.wmcloud.org/ ([[phab:T256168|T256168]])
* 21:57 bd808: Add web security group to deployment-deploy04 ([[phab:T256168|T256168]])
* 20:45 James_F: Zuul: Restrict mw*-codehealth-patch jobs to master only, for [[phab:T424573|T424573]]
* 17:16 James_F: Docker: [mediawiki-phan-taint-check-demo] Re-platform to Trixie and so PHP 8.4
* 15:53 James_F: Zuul: [mediawiki/extensions/ReportIncident] Add TestKitchen phan dependency, for [[phab:T424220|T424220]]
* 14:32 James_F: Zuul: Drop PHP 8.2 enforcement from MediaWiki things for master and REL1_46 for [[phab:T358667|T358667]]
* 12:38 Lucas_WMDE: ssh integration-castor06.integration.eqiad1.wikimedia.cloud sudo -u jenkins-deploy rm -rf /srv/castor/castor-mw-ext-and-skins/master/mwext-node24-docs-publish # fix failure seen in mwext-node24-docs-publish 383
* 09:18 James_F: jforrester@doc1004:~$ sudo -u doc-uploader rm -rf /srv/doc/cover/mediawiki-libs-node-cssjanus/ # [[phab:T424419|T424419]]

=== 2026-04-26 ===
* 20:49 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1276777

=== 2026-04-24 ===
* 22:48 dduvall: merged zuul3 branch of integration/config into master and pushed (in preparation for https://gerrit.wikimedia.org/r/c/operations/puppet/+/1277198)
* 12:27 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1276428

=== 2026-04-23 ===
* 23:57 bd808: Set `profile::beta::autoupdater::run_updater: true` for deployment-deploy04 via Horizon ([[phab:T256168|T256168]])
* 22:58 bd808: bd808@deployment-deploy04 `sudo -u jenkins-deploy /usr/local/bin/wmf-beta-update-all`
* 22:36 bd808: bd808@deployment-deploy04 `sudo -u mwdeploy /usr/local/bin/wmf-beta-update-all`
* 22:16 bd808: Disabled https://integration.wikimedia.org/ci/view/Beta/job/beta-update-databases-eqiad so that replacement script can be tested ([[phab:T256168|T256168]])
* 22:12 bd808: Disabled https://integration.wikimedia.org/ci/job/beta-code-update-eqiad so that replacement script can be tested ([[phab:T256168|T256168]])
* 22:02 bd808: Cherry-picked {{gerrit|1276813}} to deployment-puppetserver-1 ([[phab:T256168|T256168]])
* 20:11 James_F: Zuul: [wikibase/*] Replace CI testing in Node 20 with Node 24
* 20:11 James_F: Zuul: [wikidata/query/*] Replace CI testing in Node 20 with Node 24
* 20:11 James_F: Zuul: [analytics/*] Replace CI testing in Node 20 with Node 24
* 20:10 James_F: Zuul: [mediawiki/tools/*] Replace CI testing in Node 20 with Node 24
* 20:06 dancy: Upgrading gitlab cloud runners (prod) k8s from 1.34.5-do.3 to 1.35.1-do.3 ([[phab:T423726|T423726]])
* 19:55 James_F: Zuul: [jquery-client] Replace CI testing in Node 20 with Node 24
* 19:51 James_F: Zuul: [wikipeg] Drop testing in Node 20 and Node 22
* 19:47 dancy: Upgrading gitlab cloud runners (staging) k8s from 1.34.5-do.3 to 1.35.1-do.3 ([[phab:T423726|T423726]])
* 19:37 James_F: Zuul: [oojs/ui] Drop CI testing in Node 20 and Node 22
* 19:37 James_F: Zuul: [oojs/js] Drop CI testing in Node 20 and Node 22
* 19:37 James_F: Zuul: [unicodejs] Replace CI testing in Node 20 with Node 24
* 19:36 James_F: Zuul: [wikimedia/portals] Drop CI testing in Node 20 and Node 22
* 18:57 dancy: Upgrading gitlab cloud runners (prod) k8s from 1.33.9-do.3 to 1.34.5-do.3 ([[phab:T423726|T423726]])
* 18:39 dancy: Upgrading gitlab cloud runners (staging) k8s from 1.33.9-do.3 to 1.34.5-do.3 ([[phab:T423726|T423726]])
* 18:18 dancy: Upgrading gitlab cloud runners (staging) k8s from 1.33.9-do.2 to 1.33.9-do.3 ([[phab:T423726|T423726]])
* 17:58 James_F: Zuul: [mediawiki/extensions/OAuth] Add dependency on CentralAuth, for [[phab:T415281|T415281]]
* 17:56 dancy: Upgrading gitlab cloud runners (prod) k8s from 1.32.13-do.2 to 1.33.9-do.3 ([[phab:T423726|T423726]])
* 16:35 James_F: Zuul: Enforce PHP 8.5 CI for MW things in master (and REL1_46), for [[phab:T411814|T411814]]
* 16:19 James_F: Zuul: [mediawiki/services/parsoid] Enable PHP 8.5 CI
* 15:47 James_F: Zuul: [mediawiki/extensions/WikimediaCustomizations] Add AntiSpoof dependency, for [[phab:T420548|T420548]]
* 14:20 Lucas_WMDE: ssh integration-castor06.integration.eqiad1.wikimedia.cloud sudo -u jenkins-deploy rm -rf /srv/castor/castor-mw-ext-and-skins/master/mediawiki-node24 # fix failure seen in mediawiki-node24 8385 and 8405
* 12:56 James_F: Zuul: [mediawiki/extensions/GrowthExperiments] Add CentralNotice dependency, for [[phab:T422082|T422082]]

=== 2026-04-22 ===
* 00:07 James_F: Zuul: [mediawiki/extensions/DiscussionTools] Add MF dependency, for [[phab:T424113|T424113]]

=== 2026-04-21 ===
* 23:26 James_F: Zuul: [mediawiki/extensions/WikiLambda] Add CommunityConfiguration dep too, for [[phab:T394410|T394410]]
* 23:17 James_F: Zuul: [mediawiki/extensions/DiscussionTools] Add standalone test jobs, for [[phab:T422031|T422031]]
* 20:47 inflatador: updating cirrussearch hosts to Trixie/OpenSearch 2 [[phab:T421763|T421763]]
* 20:38 James_F: Zuul: [mediawiki/extensions/WikiLambda] Add CommunityConfiguration phan dep, for [[phab:T394410|T394410]]
* 20:17 bd808: Running tofu for [[phab:T421244|T421244]]
* 18:00 James_F: Zuul: [mediawiki/extensions/WatchAnalytics] Add ApprovedRevs Phan dependency
* 16:35 bd808: Unblock 79.116.0.0/16
* 13:34 James_F: Zuul: [mediawiki/extensions/WikiLambda] Add TestKitchen phan dep, for [[phab:T415254|T415254]]
* 13:27 James_F: Zuul: [mediawiki/extensions/WikimediaCustomizations] Add CentralAuth dependency, for [[phab:T420548|T420548]]

=== 2026-04-20 ===
* 23:56 bd808: Unblock 76.157.0.0/16
* 18:28 dancy: Upgrading gitlab cloud runners (staging) to 1.33.9-do.2 ([[phab:T423726|T423726]])
* 18:28 dancy: Upgrading gitlab cloud runners (staging) ([[phab:T423726|T423726]])
* 18:19 James_F: jjb: All 486 (!) jobs now updated for [[phab:T423622|T423622]]
* 18:18 bd808: Unblock 113.128.0.0/15
* 15:03 James_F: Docker: Bump ci-bullseye/-bookworm/-trixie for mirrors.wm.org removal, [[phab:T423622|T423622]]

=== 2026-04-19 ===
* 19:53 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1272752

=== 2026-04-17 ===
* 21:07 thcipriani: marking integration-agent-1080 offline for experimentation
* 19:30 thcipriani: reconfiguring castor-save-workspace-cache with https://gerrit.wikimedia.org/r/1273935
* 17:47 dancy: Upgrading gitlab cloud runners (prod) k8s from 1.32.10-do.1 to 1.32.13-do.2 ([[phab:T423726|T423726]])
* 16:49 dancy: Upgrading gitlab cloud runners (staging) k8s from 1.32.10-do.1 to 1.32.13-do.2 ([[phab:T423726|T423726]])

=== 2026-04-16 ===
* 20:49 dduvall: creating integration/zuul-jobs repo to serve as a mirror of opendev.org/zuul/zuul-jobs ([[phab:T406384|T406384]])
* 13:38 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1272711 [[phab:T423568|T423568]]
* 11:07 Silvan_WMDE: sudo -u jenkins-deploy rm -fR /srv/castor/castor-mw-ext-and-skins/master/mediawiki-node24/ # run on integration-castor06.integration.eqiad1.wikimedia.cloud

=== 2026-04-15 ===
* 20:05 James_F: Zuul: Configure REL1_46 CI, for [[phab:T423257|T423257]]
* 17:44 bd808: Unblock 176.0.0.0/13
* 17:39 bd808: Unblock 46.128.0.0/16
* 17:32 bd808: Unblock 176.86.0.0/16
* 16:39 brennen: Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/commit/127d783b2176ac60b646a5fa4f1b1a872ca66340
* 15:33 brennen: Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/100
* 01:02 brennen: Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/99

=== 2026-04-14 ===
* 20:42 James_F: Docker: [composer-scratch] Upgrade composer to 2.9.7 and cascade
* 16:35 bd808: Unblock 88.112.0.0/14
* 00:48 bd808: Unblock 24.6.0.0/16
* 00:42 bd808: Unblock 152.231.48.0/20

=== 2026-04-13 ===
* 22:00 James_F: Zuul: [mediawiki/vendor] Drop accidental Wikibase browser tests on branches
* 20:28 James_F: Zuul: [mediawiki/extensions/Chart] Drop Doxygen publish job, not used
* 14:42 James_F: Zuul: [mediawiki/extensions/WikimediaCustomizations] Add FlaggedRevs dep, for [[phab:T421011|T421011]]

=== 2026-04-12 ===
* 18:21 James_F: jforrester@contint1002:~$ sudo /usr/sbin/service zuul restart &amp;&amp; tail -f -n100 /var/log/zuul/zuul.log # [[phab:T423027|T423027]]

=== 2026-04-10 ===
* 23:22 James_F: jforrester@contint1002:~$ zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/extensions/ReadingLists --change {{Gerrit|1269498}},2 # [[phab:T422976|T422976]]
* 23:20 James_F: Zuul: [mediawiki/extensions/ReadingLists] Publish JS coverage, for [[phab:T422976|T422976]]
* 23:13 James_F: Zuul: Migrate a few straggler Node 20 MediaWiki things to Node 24
* 23:01 James_F: Zuul: Move all MediaWiki things from mediawiki-node20 to mediawiki-node24
* 21:59 James_F: Docker: Bump Node base images to March releases and cascade; Upgrade Quibble images from Node 20 to Node 24
* 10:24 hashar: Updating all Quibble jobs to 1.17.1
* 10:22 hashar: Updated PostgreSQL jobs to Quibble 1.17.1 # [[phab:T422110|T422110]]
* 10:22 hashar: Updated apitesting job to Quibble 1.17.1 # [[phab:T422843|T422843]] [[phab:T418743|T418743]]
* 09:51 hashar: Tag Quibble 1.17.1 @ {{Gerrit|0a1ab3b7c3dfee36c9bc2e9b049957d94e190e85}}

=== 2026-04-09 ===
* 15:13 hashar: Rolling back Quibble jobs to 1.16.0 (api-testing stage fails due to missing npm install step`
* 14:58 hashar: Upgrading Quibble jobs to 1.17.0
* 14:23 hashar: Tagged Quibble 1.17.0 @ {{Gerrit|864381c6b63bdbcd8c74a3162c406fffcaaf8694}}
* 07:48 hashar: Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/1268559 &quot;Zuul: use standalone jobs for GrowthExperiments Cypress tests&quot; {{!}} [[phab:T417412|T417412]]

=== 2026-04-08 ===
* 22:19 dancy: Updating docker-pkg files on contint primary for https://gerrit.wikimedia.org/r/c/integration/config/+/1269068
* 22:01 bd808: Unblock 95.216.12.170/32 ([[phab:T422751|T422751]])
* 19:26 brennen: gitlab-webhooks: building &amp; deploying https://gitlab.wikimedia.org/repos/releng/gitlab-webhooks/-/merge_requests/37 - hitting some build tooling stuff, trying a fix per instructions in the error log
* 17:54 bd808: Unblock 167.56.0.0/13 ([[phab:T422721|T422721]])
* 06:31 hashar: Deleted integration-agent-castor05 Bullseye instance, replaced by integration-agent-castor06 which is on Bookworm # [[phab:T421114|T421114]]
* 06:24 hashar: Deleted integration-agent-qemu-1003 Bullseye image, replaced by integration-agent-qemu-1004 which is on Bookworm # [[phab:T422488|T422488]]

=== 2026-04-07 ===
* 22:25 dduvall: adding new pipelinelib labels to ci nodes ([[phab:T422234|T422234]])
* 20:05 hashar: Triggered a build of https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/
* 17:06 dduvall: added `Docker` label to `contint` jenkins nodes ([[phab:T422507|T422507]])
* 17:05 dduvall: restored missing `pipelinelib` labels on `integration-agent-docker-` CI hosts ([[phab:T422507|T422507]])
* 16:53 bd808: Unblock 73.0.0.0/8 ([[phab:T422498|T422498]])
* 12:36 hashar: jjb: use $CASTOR_HOST for Quibble success cache. https://gerrit.wikimedia.org/r/1268545  {{!}}  This causes the Quibble jobs to use a new instance for the success cache, which is empty  # [[phab:T383243|T383243]]  [[phab:T421114|T421114]]
* 12:17 hashar: Migrated Castor from integration-castor05 to integration-castor06. Updated CASTOR_HOST in Jenkins and moved the Cinder volume to the new instance # [[phab:T421114|T421114]]
* 11:14 hashar: Added Bookworm based Jenkins agents to the pool Hostnames 1090, 1091, 1092 and 1093 # [[phab:T421114|T421114]]
* 10:09 hashar: Added Bookworm based Jenkins agents to the pool Hostnames 1083 to 1089 # [[phab:T421114|T421114]]
* 07:23 hashar: CI Jenkins: removed `blubber` label from all agents after having moved PipelineLib to use the `Docker` label {{!}} [[phab:T422234|T422234]]

=== 2026-04-06 ===
* 16:01 dancy: Updating docker-pkg files on contint primary for https://gerrit.wikimedia.org/r/c/integration/config/+/1268239

=== 2026-04-03 ===
* 20:17 bd808: Unblock 2.54.0.0/16 ([[phab:T422238|T422238]])
* 17:25 bd808: Unblock 31.18.0.0/16 ([[phab:T422245|T422245]])
* 17:18 bd808: Unblock 2.54.128.0/19 ([[phab:T422238|T422238]])
* 16:18 hashar: Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/1264649 &quot;add Python 3.14 to pywikibot jobs and separate lint tests&quot; {{!}} [[phab:T421723|T421723]]
* 09:26 hashar: integration: nuked pywikibot/core pre-commit cache # [[phab:T422242|T422242]]
* 09:15 hashar: Added Bookworm based Jenkins agents to the pool with label `Docker`. Hostnames are `integration-agent-docker-107*` # [[phab:T421114|T421114]]
* 02:47 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1267398

=== 2026-04-02 ===
* 16:50 thcipriani: restart jenkins
* 15:15 bd808: Unblock 82.216.0.0/16 ([[phab:T421508|T421508]])
* 15:07 bd808: Unblock 95.90.0.0/15 ([[phab:T421485|T421485]])
* 11:19 James_F: Zuul: [oojs/ui] Drop ooui-ruby2.7-rake job, we're abandoning Ruby use there

=== 2026-04-01 ===
* 22:01 bd808: Unblock 109.144.0.0/12 ([[phab:T422019|T422019]])
* 20:16 bd808: Unblock 93.192.0.0/10 ([[phab:T421894|T421894]])
* 19:25 dancy: Updating buildkitd to v0.29.0 in gitlab-cloud-runners (prod) ([[phab:T415284|T415284]])
* 17:57 brennen: Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/97 ([[phab:T420441|T420441]])
* 17:39 bd808: Unblock 94.134.0.0/15 ([[phab:T421866|T421866]])
* 16:31 dancy: Upgrade buildkit to 0.29.0 in staging gitlab-cloud-runners ([[phab:T415284|T415284]])
* 10:47 taavi: integration-castor05: free up a bit of disk space by deleting cache for AhoCorasick/ CLDRPluralRuleParser/ HtmlFormatter/ RelPath/ RunningStat/ IPSet/

=== 2026-03-30 ===
* 22:01 bd808: Unblock 78.20.0.0/14 ([[phab:T421586|T421586]])
* 21:04 bd808: Unblock 95.88.0.0/15 ([[phab:T421774|T421774]])
* 20:49 bd808: Unblock 95.89.191.0/24 ([[phab:T421774|T421774]])
* 20:29 bd808: Unblock 73.162.0.0/16 ([[phab:T421549|T421549]])
* 13:10 hashar: gerrit: abandon mediawiki/core changes that are 2+years old and are attached to a task (`Bug: Txxxx`)
* 11:37 hashar: Reloaded Zuul to to add 3 persons to the  allow list
* 10:43 James_F: Docker: Re-pushing to try to create quibble-coverage 1.16.0-s2

=== 2026-03-27 ===
* 21:00 James_F: Docker: [quibble-bullseye] Drop Python 2 from images
* 11:28 hashar: deployment-prep: removed block for `143.176.0.0/15` and blocked subblock `143.176.0.0/16` instead. This unblocks `143.177.0.0/16` # [[phab:T421420|T421420]]
* 00:18 bd808: Unblock 95.90.238.0/23 ([[phab:T421447|T421447]])

=== 2026-03-26 ===
* 21:25 bd808: Unblock 89.240.0.0/15 ([[phab:T421364|T421364]])
* 21:09 brennen: patchdemo: deploy to production for https://gitlab.wikimedia.org/repos/test-platform/catalyst/patchdemo/-/merge_requests/312

=== 2026-03-25 ===
* 20:41 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1256318 [[phab:T421283|T421283]]
* 15:46 dancy: Migrated gitlab-cloud-runners (prod) from nginx-ingress to traefik ([[phab:T420743|T420743]])
* 15:32 dancy: Migrated gitlab-cloud-runners (staging) from nginx-ingress to traefik ([[phab:T420743|T420743]])
* 10:01 hashar: Updating tox Jenkins jobs to add support for Python 3.14 {{!}} https://gerrit.wikimedia.org/r/1260632 {{!}} [[phab:T421209|T421209]]
* 08:40 codders: integration: integration-castor05: rm -fR /srv/castor/castor-mw-ext-and-skins/master/mediawiki-node20/

=== 2026-03-24 ===
* 19:40 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1255746
* 15:34 brennen: gitlab1004: manual test run of `configure-projects` with cleared issue allowlist ([[phab:T412882|T412882]])
* 15:26 bd808: Unblock 47.194.0.0/16 ([[phab:T421127|T421127]])
* 12:53 hashar: integration: deleted old Puppet 5 compiler agents from Jenkins ( pcc-worker1014.puppet-diffs.eqiad1.wikimedia.cloud , pcc-worker1015.puppet-diffs.eqiad1.wikimedia.cloud , pcc-worker1016.puppet-diffs.eqiad1.wikimedia.cloud ) # [[phab:T367399|T367399]]
* 07:42 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1259755

=== 2026-03-23 ===
* 15:28 Lucas_WMDE: ssh integration-castor05.integration.eqiad1.wikimedia.cloud sudo -u jenkins-deploy rm -rf /srv/castor/castor-mw-ext-and-skins/master/mediawiki-node20 # fix failure seen in mediawiki-node20 90272

=== 2026-03-22 ===
* 14:52 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1258082
* 01:00 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1256488

=== 2026-03-21 ===
* 08:10 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1256962
* 07:48 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1256946

=== 2026-03-20 ===
* 21:21 bd808: Unblock 103.159.218.0/24 ([[phab:T420530|T420530]])
* 14:59 James_F: Zuul: [mediawiki/extensions/AbuseFilter] Add dependency on CodeMirror, for [[phab:T399673|T399673]]

=== 2026-03-19 ===
* 16:54 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1255777
* 16:01 Krinkle: Hoist l10n-bot rights from labs/tools parent to labs parent to reduce duplication in other labs/ repos
* 15:50 Krinkle: Create labs/xtools repo (branch: main, parent: labs, owner: labs-xtools), ref [[phab:T402086|T402086]]

=== 2026-03-18 ===
* 21:11 dcausse: [[phab:T403775|T403775]]: reindexing all wikis to enable new sorting options
* 21:08 dcausse: restarting opensearch on deployment-cirrussearch(12{{!}}13{{!}}14) instances to pickup new plugin versions
* 14:56 James_F: Zuul: Handle wmf/next the same way as wmf/branch_cut_pretest
* 14:52 James_F: Zuul: [GrowthExperiments] drop duplicate VisualEditor dep
* 14:52 James_F: Zuul: [search/*] Add experimental Java 25 jobs

=== 2026-03-17 ===
* 22:50 James_F: Zuul: [mediawiki/extensions/JsonForms] Add quibble jobs
* 21:27 James_F: Zuul: search: Update opensearch plugins for Java 11/17, for [[phab:T420407|T420407]]
* 20:20 bd808: Resize deployment-sessionstore06 from g4.cores1.ram2.disk20 to g4.cores2.ram4.disk20 ([[phab:T415021|T415021]])
* 16:43 James_F: Zuul: [BlueSpicePermissionManager] Add …ConfigManager &amp; …UserManager deps
* 14:36 James_F: Zuul: [mediawiki/extensions/ArticleGuidance]: Add SpamBlacklist as phan dep, for [[phab:T420015|T420015]]

=== 2026-03-13 ===
* 13:59 andrewbogott: deleting ptr record 117.0.16.172.in-addr.arpa. -- accidental duplicate for deployment-kafka-logging01.deployment-prep.eqiad1.wikimedia.cloud
* 13:04 elukey: re-create kafka-logging-01 in deployment-prep on trixie and Kafka 3.7 (was running on buster)
* 09:13 elukey: upgrade kafka-jumbo and kafka-main to Confluent 7.7 in deployment-prep (pre-requisite before being able to upgrade to Trixie)

=== 2026-03-12 ===
* 21:23 bd808: Hard reboot deployment-sessionstore06 ([[phab:T415021|T415021]])
* 01:14 James_F: Docker: [helm-linter] Bump for Envoy 1.35.9, for [[phab:T419637|T419637]]

=== 2026-03-11 ===
* 16:48 James_F: jforrester@doc1004:~$ sudo -u doc-uploader rm -rf /srv/doc/cover-extensions/MetricsPlatform # [[phab:T417568|T417568]]
* 16:47 James_F: Zuul: [mediawiki/extensions/MetricsPlatform] Archive, for [[phab:T416865|T416865]]
* 11:12 hashar: Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/1250529 &quot;inference-services: Split policy violation CI into separate model jobs.&quot; - [[phab:T418832|T418832]]

=== 2026-03-10 ===
* 17:39 dduvall: deployed reggie v1.18.0 to gitlab-cloud-runner production
* 17:11 hashar: Updated MediaWiki coverage jobs so that they now keep &quot;Generate a local configuration by running `composer phpunit:config`&quot; message # [[phab:T419073|T419073]]
* 16:41 dduvall: deployed reggie v1.18.0 to gitlab-cloud-runner staging
* 08:21 codders: integration: integration-castor05: rm -fR /srv/castor/castor-mw-ext-and-skins/master/mediawiki-node20

=== 2026-03-09 ===
* 21:53 bd808: Reboot deployment-shellbox01 on the off chance that is makes the new permissions error go away ([[phab:T419440|T419440]])
* 13:13 James_F: Zuul: [mediawiki/extensions/WikiShare] Mark as archived, for [[phab:T413589|T413589]]
* 13:11 James_F: Zuul: [mediawiki/extensions/Memento] Mark as archived, for [[phab:T369991|T369991]]
* 13:10 James_F: Zuul: [mediawiki/extensions/QuickGV] Mark as archived, for [[phab:T413348|T413348]]
* 13:10 James_F: Zuul: [mediawiki/extensions/SemanticImageInput] Mark as archived, for [[phab:T413588|T413588]]
* 13:09 James_F: Zuul: [mediawiki/extensions/SidebarDonateBox] Mark as archived, for [[phab:T413587|T413587]]
* 13:07 James_F: Zuul: [mediawiki/extensions/SemanticSifter] Mark as archived, for [[phab:T413586|T413586]]
* 13:06 James_F: Zuul: [mediawiki/extensions/GoogleAdSense] Mark as archived, for [[phab:T413585|T413585]]
* 13:04 James_F: Zuul: [mediawiki/extensions/SecurityAPI] Mark as archived, for [[phab:T418008|T418008]]
* 12:50 James_F: Zuul: [mediawiki/extensions/CheckUser] Add DiscussionTools dependency
* 12:50 James_F: Zuul: [mediawiki/skins/MinervaNeue] Add dependencies for TestKitchen
* 10:40 hashar: gerrit: mediawiki/vendor: converted `es6` and `es710` branches to tags # [[phab:T417804|T417804]]
* 09:24 hashar: Updating Quibble jobs to 1.16.0 {{!}} https://gerrit.wikimedia.org/r/c/integration/config/+/1248880 {{!}} [[phab:T417399|T417399]] [[phab:T417409|T417409]] [[phab:T418461|T418461]]
* 09:15 hashar: updating all CI Jenkins jobs using `./jjb-update`

=== 2026-03-06 ===
* 19:46 James_F: Zuul: [mediawiki/services/geoshapes] Mark as archived, for [[phab:T418372|T418372]]
* 16:37 hashar: Building Docker images for Quibble 1.16.0
* 16:31 hashar: Tag Quibble 1.16.0 @ {{Gerrit|0b9db5fe3cabb2cec0b5d44e128bafa917b3b895}} # [[phab:T417399|T417399]] [[phab:T417409|T417409]] [[phab:T418461|T418461]]
* 12:32 hashar: Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/1248411 &quot;jjb, Zuul: vary Wikibase Selenium for release branches&quot; {{!}} [[phab:T418797|T418797]]
* 12:12 hashar: Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/1248409/ &quot;jjb, Zuul: rename wikibase-selenium job for clarity&quot; {{!}} [[phab:T418797|T418797]]

=== 2026-03-05 ===
* 14:41 James_F: Zuul: [mediawiki/skins/MinervaNeue] Add TestKitchen as a dependency for [[phab:T418053|T418053]]
* 08:01 hashar: Reloaded Zuul to rename wikibase-client / wikibase-repo jobs {{!}} https://gerrit.wikimedia.org/r/1238317
* 00:04 James_F: Docker: [quibble-coverage] Use local PHPUnit config, for [[phab:T345481|T345481]]

=== 2026-03-04 ===
* 21:16 James_F: Zuul: [mediawiki/core] Make PHP 8.5 voting on master branch, for [[phab:T411814|T411814]]
* 21:10 James_F: Zuul: [mediawiki/vendor] Make PHP 8.5 voting on master branch, for [[phab:T411814|T411814]]
* 19:48 brennen: Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/96 ([[phab:T419004|T419004]])
* 18:50 James_F: Revert &quot;Zuul: [mediawiki/extensions/MobileFrontend] Add ParserMigration dependency&quot;, for [[phab:T419043|T419043]]
* 16:23 James_F: Zuul: [mediawiki/services/parsoid] Make PHP 8.4 voting
* 15:37 James_F: Docker: [rake-ruby2.7] Add libffi-dev too, for [[phab:T418463|T418463]]
* 13:59 James_F: Docker: [rake-ruby2.7] Add ruby-ffi for [[phab:T418463|T418463]]
* 13:54 hashar: SIGKILL Zuul cause it can't gracefully stop most probably due to being locked attempting to report back to Gerrit # [[phab:T419009|T419009]]
* 13:49 hashar: Stopping Zuul # [[phab:T419009|T419009]]
* 13:41 hashar: Took a Zuul stack dump on contint1002.wikimedia.org using SIGUSR1 # [[phab:T419009|T419009]]

=== 2026-03-03 ===
* 23:52 James_F: Zuul: [mediawiki/extensions/WikimediaMessages] Drop MetricsPlatform phan dep
* 23:52 James_F: Zuul: [mediawiki/extensions/WikimediaEvents] Drop MetricsPlatform phan dep

=== 2026-03-02 ===
* 22:13 James_F: Zuul: Enforce PHP 8.4 in MW extensions and skins for development branch, for [[phab:T386108|T386108]]
* 14:05 James_F: Zuul: [mediawiki/extensions/MobileFrontend] Add ParserMigration dependency, for [[phab:T415451|T415451]]
* 13:48 James_F: Zuul: […/WikimediaEvents] Drop LoginNotify dependency, now unused, for [[phab:T404334|T404334]]
* 10:16 Lucas_WMDE: ssh integration-castor05.integration.eqiad1.wikimedia.cloud sudo -u jenkins-deploy rm -rf /srv/castor/castor-mw-ext-and-skins/master/quibble-vendor-mysql-php83-selenium/Cypress/15.8.2/ # [[phab:T418718|T418718]]

=== 2026-02-28 ===
* 21:33 hashar: gerrit: triggering replication to GitHub for all of `mediawiki/skins`  # [[phab:T418675|T418675]]
* 21:33 hashar: gerrit: triggering replication to GitHub for all of `mediawiki/extensions`  # [[phab:T418675|T418675]]

=== 2026-02-27 ===
* 15:53 dancy: Updating gitlab-cloud-runners (staging and prod) to gitlab-runner 18.9.0.

=== 2026-02-26 ===
* 20:16 James_F: Zuul: Provide a custom, high-priority pipeline just for puppet compiler [[phab:T414621|T414621]]
* 19:32 James_F: Docker: Bump all the PHPs.
* 13:40 hashar: Deployed Jenkins job https://integration.wikimedia.org/ci/job/wikibase-selenium/ # [[phab:T287582|T287582]]
* 00:13 dduvall: forcing replacement of buildkitd helm release in gitlab-cloud-runner prod cluster due to dependency on removed k8s secret ([[phab:T416260|T416260]])

=== 2026-02-25 ===
* 23:50 dduvall: deploying https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/552 to gitlab-cloud-runner production cluster ([[phab:T416260|T416260]])
* 14:07 James_F: Zuul: [mediawiki/extensions/CommunityRequests] Add TemplateData dependency, for [[phab:T401638|T401638]]
* 00:08 jeena: no-op testing updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/95

=== 2026-02-24 ===
* 15:55 brennen: devtools: test deploy phab/phorge to test instance ([[phab:T418256|T418256]])

=== 2026-02-23 ===
* 23:07 jeena: Updated development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/92
* 22:43 dancy: Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/92
* 22:12 bd808: Unblock 191.80.192.0/18 ([[phab:T418132|T418132]])
* 20:26 hashar: Deleted &quot;replication-upstream&quot; Grafana dashboard in favor of a copy/new &quot;replication&quot; one.  https://grafana.wikimedia.org/d/RFLS1GsWk/replication-upstream , replaced it by https://grafana.wikimedia.org/d/d4a4da73-c27f-4ce6-a9e5-ab84dd7a4ebb/replication
* 16:29 James_F: Zuul: [3d2png] Add basic Node CI at version 20

=== 2026-02-20 ===
* 21:47 bd808: Unblock 168.184.84.0/24 ([[phab:T418020|T418020]])
* 17:13 bd808: Unblock 122.187.64.0/18 ([[phab:T417964|T417964]])
* 14:35 James_F: Zuul: [mediawiki/extensions/Monstranto] Move out of Wikimedia prod section

=== 2026-02-19 ===
* 18:34 bd808: Unblock 181.98.0.0/16 ([[phab:T417890|T417890]])
* 17:21 James_F: Zuul: [mediawiki/extensions/WikimediaEvents] Add AbuseFilter as a dependency, for [[phab:T417799|T417799]]
* 13:22 hashar: Reloaded Zuul to archive the Cergen repository {{!}} https://gerrit.wikimedia.org/r/c/integration/config/+/1240688 {{!}} [[phab:T417887|T417887]]

=== 2026-02-18 ===
* 20:17 jeena: Updating development images on contint primary for [[phab:T415922|T415922]]
* 19:44 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1240360
* 18:40 bd808: Unblock 46.59.0.0/17 ([[phab:T417747|T417747]])
* 17:05 hashar: Regenerating Jenkins jobs with JJB based on https://gerrit.wikimedia.org/r/c/integration/config/+/1240254/
* 17:04 hashar: Added EXT_DEPENDENCIES to Quibble Jenkins jobs parameters so we can manually trigger them from the Web UI using a different set of deps # https://gerrit.wikimedia.org/r/c/integration/config/+/1240254/
* 16:30 hashar: Triggered https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/ with empty Zuul parameters introduced by https://gerrit.wikimedia.org/r/1240333 {{!}} https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/4893/console
* 15:43 James_F: Zuul: [mediawiki/extensions/ReadingLists] Add EventBus dependency for [[phab:T417706|T417706]]
* 12:15 hashar: zuul-1001.zuul3.eqiad1.wikimedia.cloud: added keepalive=20 to the scheduler Gerrit driver and restarted scheduler container # [[phab:T417497|T417497]]
* 06:58 jeena: Updating development images on contint primary for [[phab:T415922|T415922]]

=== 2026-02-17 ===
* 23:37 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1240081
* 23:20 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1240078
* 15:58 brennen: deployed latest phab/phorge wmf/stable to devtools test instance ([[phab:T417657|T417657]])
* 09:01 hashar: Reloaded Zuul to enable php 8.5 testing on utfnormal, php-session-serializer, wikipeg, mediawiki/libs/Dodo, mediawiki/libs/UUID, testing-access-wrapper and translatewiki # [[phab:T406326|T406326]]

=== 2026-02-16 ===
* 15:27 hashar: Manually cleaned some old workspaces on integration-agent-docker-1042

=== 2026-02-12 ===
* 20:07 James_F: Zuul: Enable PHP 8.5 jobs for most MW libraries, for [[phab:T406326|T406326]]
* 19:33 James_F: Docker: [php83] Re-build with upstream's new 8.3.30 release and cascade
* 19:31 James_F: Zuul: Add PHP 8.5 CI job to various things noted as blocked by Phan, for [[phab:T410941|T410941]], [[phab:T406326|T406326]]
* 16:35 Krinkle: Disable publishing noise on tasks from repos Bcp47, clover-diff, ScopedCallback, and IDLeDOM. Ref [[phab:T143162|T143162]]
* 15:53 dancy: Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/87
* 11:21 James_F: Zuul: [mediawiki/libs/shellbox] Add direct Phan job, for [[phab:T416064|T416064]]

=== 2026-02-10 ===
* 20:16 dancy: Rebooted k3s.catalyst-dev (it was unresponsive, but the reboot hasn't helped)

=== 2026-02-09 ===
* 21:58 James_F: Zuul: [mediawiki/tools/phan] Add PHP 8.5 CI job, for [[phab:T410941|T410941]]
* 19:46 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1238006 [[phab:T415680|T415680]]
* 11:51 James_F: Zuul: [mediawiki/extensions/ReadingLists] Drop MetricsPlatform dependency, for [[phab:T414435|T414435]]

=== 2026-02-05 ===
* 17:58 James_F: Zuul: […/WikimediaCustomizations] Add six new dependencies for [[phab:T404334|T404334]]
* 15:35 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1237254
* 15:18 James_F: Zuul: […/OATHAuth] Add dependency and phan dependency on CentralAuth

=== 2026-02-04 ===
* 12:54 James_F: Zuul: [mediawiki/extensions/Petition] Add CLDR dependency
* 10:03 hashar: Restarted Jenkins on releases2003.codfw.wmnet

=== 2026-02-02 ===
* 21:17 hashar: Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/1234926 &quot;re-enable master jobs for some BlueSpice repos - [[phab:T403196|T403196]]&quot;
* 21:05 bd808: Unblock 85.146.0.0/17 ([[phab:T416079|T416079]])
* 19:47 James_F: Zuul: […/WikimediaCustomizations] Add cldr phan dependency, for [[phab:T404334|T404334]]
* 17:33 bd808: Unblock 188.188.0.0/15 ([[phab:T416095|T416095]])
* 17:26 bd808: Unblock 85.94.84.0/22 ([[phab:T416105|T416105]])
* 17:09 bd808: Unblock 94.234.0.0/16 ([[phab:T416165|T416165]])
* 16:51 dancy: Update gitlab-runners to alpine-v18.6.6 ([[phab:T415214|T415214]])
* 16:27 bd808: Unblock 47.231.208.0/21 ([[phab:T416010|T416010]])
* 11:39 James_F: Zuul: […/WikimediaCustomizations] Add five new phan dependencies, for [[phab:T404334|T404334]]
* 09:45 Lucas_WMDE: ssh integration-castor05.integration.eqiad1.wikimedia.cloud sudo -u jenkins-deploy rm -rf /srv/castor/castor-mw-ext-and-skins/master/mediawiki-node20 # fix failure seen in mediawiki-node20 58532, 58557

=== 2026-01-31 ===
* 21:49 James_F: Deleted Jenkins's job entry for castor-save-workspace-cache {{Gerrit|6193776}} and this seems to have unstuck things for [[phab:T416078|T416078]]?
* 21:45 James_F: Running `sudo systemctl restart jenkins` on contint for [[phab:T416078|T416078]]
* 21:44 James_F: Fighting [[phab:T416078|T416078]], took integration-castor-5 offline, disconnected, sshed in to kill threads, then reconnected; no change in aspect.
* 19:03 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1235380

=== 2026-01-28 ===
* 21:26 James_F: jforrester@doc1004:~$ sudo -u doc-uploader rm -rf /srv/doc/cover-extensions/WebAuthn # [[phab:T415832|T415832]]
* 21:11 bd808: Unblock 181.160.0.0/15 &amp; 186.40.128.0/17 ([[phab:T415820|T415820]])
* 17:01 bd808: Unblock 102.182.0.0/16 ([[phab:T415782|T415782]])

=== 2026-01-27 ===
* 16:45 James_F: Zuul: Switch skin-quibble template with identical extension-quibble, for [[phab:T402398|T402398]]
* 16:18 James_F: Zuul: [ArticleGuidance] mention it will be in production
* 15:55 James_F: Docker: [quibble-bullseye] Update to Quibble 1.15.0
* 15:12 James_F: Docker: [quibble-coverage] Pass PHPUnit config location explicitly, for [[phab:T395470|T395470]]
* 09:18 hashar: integration: on integration-castor05, deleted caches for old MediaWiki branches
* 09:15 hashar: integration: on pkgbuilder instances, removed Buster cow images, aptcache and hooks. `sudo cumin --force -p 0 'name:pkgbuilder' 'rm -fR /srv/pbuilder/&lt;nowiki&gt;{&lt;/nowiki&gt;base-buster-amd64.cow,hooks/buster,aptcache/buster-amd64&lt;nowiki&gt;}&lt;/nowiki&gt;'`  # [[phab:T397209|T397209]]
* 09:14 hashar: integration: cleaned up old workspaces under /srv/jenkins/workspace

=== 2026-01-26 ===
* 23:27 bd808: Unblock 66.130.0.0/15 ([[phab:T415596|T415596]])
* 22:52 bd808: Unblock 45.16.0.0/12 ([[phab:T415467|T415467]])
* 14:46 hashar: gerrit: changed `operations/software/permissions` project type from `CODE` to `PERMISSIONS` by pointing `HEAD` to `refs/meta/config`

=== 2026-01-22 ===
* 17:36 James_F: Docker:  [quibble-coverage] Stop using legacy PHPUnit entrypoint ([[phab:T395470|T395470]]) &amp; Stop excluding Dump/ParserFuzz/Stub groups ([[phab:T415230|T415230]])
* 15:11 James_F: Zuul: [mediawiki/extensions/Math] Add a standalone job, for [[phab:T415230|T415230]]

=== 2026-01-20 ===
* 20:38 bd808: Cherry picked https://gerrit.wikimedia.org/r/c/operations/puppet/+/1229186 ([[phab:T415113|T415113]])
* 19:05 bd808: Rebooted deployment-cache-text08 to see if the mystery haproxy startup failure would go away ([[phab:T415100|T415100]])
* 18:50 bd808: Unblock 152.7.0.0/16 ([[phab:T415100|T415100]])

=== 2026-01-17 ===
* 23:32 ori: beta-scap with `php_l10n: true` completed successfully: https://integration.wikimedia.org/ci/view/Beta/job/beta-scap-sync-world/241466/console. PHP l10n files generated. Reverted local change to scap.cfg.
* 23:26 ori: Temporarily set `php_l10n: true` on deployment-deploy04:/etc/scap.cfg to see if next scap succeeds.

=== 2026-01-16 ===
* 16:33 dancy: Deleting deployment-mx03.deployment-prep ([[phab:T412975|T412975]])

=== 2026-01-15 ===
* 14:50 James_F: jforrester@doc1004:~$ sudo -u doc-uploader rm -rf /srv/doc/cover-extensions/ArticleSummaries/ # [[phab:T413232|T413232]]

=== 2026-01-14 ===
* 17:14 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1226907
* 16:27 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1226893
* 15:57 bd808: Unblock 190.60.63.0/24 ([[phab:T414541|T414541]])

=== 2026-01-13 ===
* 15:04 James_F: Zuul: Make quibble-for-mediawiki-core-vendor-mysql-php84 voting, for [[phab:T386108|T386108]]

=== 2026-01-12 ===
* 21:33 zabe: zabe@deployment-mwmaint03:~$ foreachwiki migrateLinksTable.php --table imagelinks # [[phab:T413668|T413668]]
* 21:06 bd808: Unblock 66.81.168.0/21 ([[phab:T414303|T414303]])
* 17:42 dancy: Turned off instance deployment-prep.deployment-mx03
* 11:44 Lucas_WMDE: ssh integration-castor05.integration.eqiad1.wikimedia.cloud sudo -u jenkins-deploy rm -rf /srv/castor/castor-mw-ext-and-skins/master/mediawiki-node20 # fix failure seen in mediawiki-node20 46331, 46344

=== 2026-01-10 ===
* 21:48 taavi: reload zuul for https://gerrit.wikimedia.org/r/1224782
* 00:25 bd808: Unblock 91.160.0.0/12 ([[phab:T414190|T414190]])

=== 2026-01-09 ===
* 17:33 thcipriani: re-enabling beta update jobs after test bad extension-list [[phab:T411516|T411516]]
* 17:09 thcipriani: disabling beta update jobs to test bad extension-list [[phab:T411516|T411516]])

=== 2026-01-08 ===
* 21:30 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1224815 [[phab:T414136|T414136]]
* 18:24 bd808: Unblock 89.80.0.0/12 ([[phab:T414113|T414113]])
* 15:55 dancy: Upgrading gitlab-runner to v18.5.0 on gitlab-cloud-runners. ([[phab:T414053|T414053]])

=== 2026-01-07 ===
* 23:17 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1082574 https://gerrit.wikimedia.org/r/1224157 https://gerrit.wikimedia.org/r/1224159
* 23:12 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/896311 [[phab:T27482|T27482]]
* 23:06 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1224218
* 17:34 James_F: Zuul: Add new extensions: IssueTrackerLinks, PreviewLinks, and WikiRAG
* 17:34 James_F: Zuul: [labs/tools/heritage] Point to the task to drop 8.1 testing
* 15:09 James_F: Zuul: [labs/tools/heritage] Add testing in PHP 8.2+, not just PHP 8.1
* 15:03 James_F: Zuul: Even for extension-broken, don't offer PHP 8.1 testing
* 15:02 James_F: Zuul: Move quibble experimental sqlite/postgres tests to PHP 8.3

=== 2026-01-06 ===
* 16:57 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1223690 [[phab:T411814|T411814]]
* 16:16 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1223189 [[phab:T411814|T411814]]
* 00:30 bd808: Unblock 85.134.128.0/17 ([[phab:T413755|T413755]])
* 00:02 bd808: Unblock 89.166.128.0/17 ([[phab:T413702|T413702]])

=== 2026-01-05 ===
* 23:57 bd808: Unblock 185.233.104.0/22 ([[phab:T413472|T413472]])
* 23:51 bd808: Unblock 45.62.112.0/21 ([[phab:T413079|T413079]])
* 23:44 bd808: Unblock 85.134.200.0/21 ([[phab:T413067|T413067]])
* 19:03 dancy: Updated buildkitd to v0.26.3 in gitlab-cloud-runners
* 14:27 taavi: reload zuul for {{Gerrit|1223191}}
* 13:57 James_F: Zuul: [mediawiki/php/wmerrors] Enable PHP 8.5 testing, for [[phab:T410921|T410921]]

=== 2026-01-03 ===
* 17:59 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1222709 https://gerrit.wikimedia.org/r/1220388 https://gerrit.wikimedia.org/r/1219140

=== 2026-01-02 ===
* 17:10 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1222597

=== 2026-01-01 ===
* 02:34 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1221644

&lt;noinclude&gt;'''Server Admin Log''' logged from {{IRC|wikimedia-releng}} for [[Nova Resource:Deployment-prep|Beta Cluster]], [[mw:Continuous integration|Continuous integration]] and various other Release Engineering projects.&lt;/noinclude&gt;

{{SAL-archives/Release Engineering}}

&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>jjj35r9gxfs24i69qt9q19ma3denzlr</sha1>
    </revision>
  </page>
  <page>
    <title>Nova Resource:Tools.cluebotng/SAL</title>
    <ns>498</ns>
    <id>443628</id>
    <revision>
      <id>2408870</id>
      <parentid>2408661</parentid>
      <timestamp>2026-05-04T07:27:41Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25252392662 (https://github.com/cluebotng/component-configs/commits/7352cd4f730ca9f5c276772f0b338230989feef4)</comment>
      <origin>2408870</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="14437" sha1="0eu8u8r93nalmth1qxefvgwwzl7asa8" xml:space="preserve">=== 2026-05-04 ===
* 07:27 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25252392662 (https://github.com/cluebotng/component-configs/commits/7352cd4f730ca9f5c276772f0b338230989feef4)

=== 2026-05-02 ===
* 12:57 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/25252392662 (https://github.com/cluebotng/component-configs/commits/7352cd4f730ca9f5c276772f0b338230989feef4)

=== 2026-04-24 ===
* 22:18 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/24914278570 (https://github.com/cluebotng/component-configs/commits/23a4b53f3d291b0c750d44a2c0a661333307786d)
* 00:19 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24865518502 (https://github.com/cluebotng/component-configs/commits/6e953f35fdca38226cde9ea7280f948f34242881)

=== 2026-04-23 ===
* 23:28 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24863962598 (https://github.com/cluebotng/component-configs/commits/07b9ad8616853af2ed49f96de6e55c14e3faabe0)
* 09:23 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24827343249 (https://github.com/cluebotng/component-configs/commits/218a7fa56222cf3e98b642eebbf6b1b2b273a92d)
* 09:12 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24826859245 (https://github.com/cluebotng/component-configs/commits/22ea2eb955d25f4a15e6b234e72a24bc01127a79)
* 09:07 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24826656498 (https://github.com/cluebotng/component-configs/commits/90aa39b3e81f19f42d08d5ec6131ba09c68bd786)

=== 2026-04-17 ===
* 00:12 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24540678933 (https://github.com/cluebotng/component-configs/commits/26849735bbefbe218cbe0ce41db5a35941798c7b)

=== 2026-04-14 ===
* 21:05 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24422836823 (https://github.com/cluebotng/component-configs/commits/10f4f0f81e169fac55d056176a273966c8160078)

=== 2026-04-10 ===
* 15:26 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24250442412 (https://github.com/cluebotng/component-configs/commits/bfa8b761a017e9b8bb69ae52c5cb731d17bd324f)
* 15:15 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24249897963 (https://github.com/cluebotng/component-configs/commits/68514222ba9a90ece524baf75b02c9835faf87d3)
* 14:25 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24247614873 (https://github.com/cluebotng/component-configs/commits/30bda68a3ea7a1674d174e43cc8651d301c7485c)
* 14:23 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24247570676 (https://github.com/cluebotng/component-configs/commits/93ced49392782bf65e34d13f10cbeaafa760f115)

=== 2026-04-09 ===
* 18:17 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24206093216 (https://github.com/cluebotng/component-configs/commits/a97bfe791582e24f1c696f1bd89b965ea233c253)

=== 2026-03-27 ===
* 17:44 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/23659656823 (https://github.com/cluebotng/component-configs/commits/f4a494492433360a06326a918985c51c6d0828d4)
* 17:42 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/23659535101 (https://github.com/cluebotng/component-configs/commits/b825610c7f23870a8561b00b5f8546b107643015)

=== 2026-03-25 ===
* 10:14 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/23535729395 (https://github.com/cluebotng/component-configs/commits/c1468d960041cd66ab50902f344fec1ac65ddcad)

=== 2026-03-21 ===
* 16:26 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/23383557597 (https://github.com/cluebotng/component-configs/commits/0b67216b47074dd5d1d279dde0aa9144b243cf01)
* 16:21 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/23383640340 (https://github.com/cluebotng/component-configs/commits/3497a25c3d209bdf8f64f3ec3e77e52f2f8debfa)
* 16:18 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/23383557597 (https://github.com/cluebotng/component-configs/commits/0b67216b47074dd5d1d279dde0aa9144b243cf01)
* 16:16 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/23383551615 (https://github.com/cluebotng/component-configs/commits/ffff74b90a37a0c6bdd565128d3c11ae195e0763)
* 14:26 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/23381633193 (https://github.com/cluebotng/component-configs/commits/ac40c461942f4541b640a32ff0141268418abc12)

=== 2026-03-19 ===
* 20:30 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/23315406679 (https://github.com/cluebotng/component-configs/commits/fd07020c08545c83ab35667616a26081966648df)

=== 2026-02-15 ===
* 10:56 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/22034380291 (https://github.com/cluebotng/component-configs/commits/842b50dc5d3160000352a25c5fdf09ea88ebf3eb)

=== 2025-11-11 ===
* 15:42 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270642865 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c)
* 15:36 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/19270642865 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c)
* 15:30 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270301650 (https://github.com/cluebotng/component-configs/commits/f28dcaec8c5882b4a1b7d861fe7f5e400312a5b4)
* 13:20 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19266856634 (https://github.com/cluebotng/component-configs/commits/b0e9170597a778654185be762c580e2a6e19492f)

=== 2025-11-06 ===
* 01:06 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19121350626 (https://github.com/cluebotng/component-configs/commits/80f69f0ab7b09c2e3e5e208d847a954cb1975bc6)

=== 2025-11-05 ===
* 19:37 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19113995211 (https://github.com/cluebotng/component-configs/commits/586f2c46dcbbb09a9f7926e991bc5fbe45f4a1e9)
* 18:41 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19112506240 (https://github.com/cluebotng/component-configs/commits/3f51ec3aa53d1378883a9dc973716e57c283d26c)
* 16:28 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19108852168 (https://github.com/cluebotng/component-configs/commits/24f3dc9fe5e2211d861c754a4b9342a6127f4a4a)
* 12:39 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19102217917 (https://github.com/cluebotng/component-configs/commits/24f3dc9fe5e2211d861c754a4b9342a6127f4a4a)

=== 2025-10-29 ===
* 15:19 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18912872623 (https://github.com/cluebotng/component-configs/commits/3281794d8d1d2e17d9e9859c6f6f7ae3c5216eda)

=== 2025-10-23 ===
* 12:29 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18748267293 (https://github.com/cluebotng/component-configs/commits/bc8f1b883d0d53edf08bea5e5319ee7ee0b4fb82)

=== 2025-10-08 ===
* 15:47 wmbot~damian-scripts@tools-bastion-15: bot deployed @ refs/tags/v1.4.1
* 11:23 wmbot~damian-scripts@tools-bastion-15: bot deployed @ refs/tags/v1.4.0

=== 2025-09-29 ===
* 16:41 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18104101448 (https://github.com/cluebotng/component-configs/commits/c49408a6e0285932adef0b5cc39e15d06c8742f5)
* 09:33 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18092350259 (https://github.com/cluebotng/component-configs/commits/283965c9240c0c5a72e0ea1203439583935295cb)

=== 2025-09-27 ===
* 13:07 wmbot~damian-scripts@tools-bastion-15: bot deployed @ refs/tags/v1.3.0

=== 2025-09-26 ===
* 18:54 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18046649036 (https://github.com/cluebotng/component-configs/commits/07b907ff75f0289f350549bae5e75bf4e91c91ca)
* 12:52 wmbot~damian-scripts@tools-bastion-15: bot deployed @ refs/tags/v1.2.4
* 12:46 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18038096319 (https://github.com/cluebotng/component-configs/commits/e10e601b4fb06b3fd97856ef86a30e5391fb4f17)
* 12:40 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18037926507 (https://github.com/cluebotng/component-configs/commits/4950150f14c22c0a7d3df1739fa5537aeba4157d)
* 12:10 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18037156332 (https://github.com/cluebotng/component-configs/commits/a51fe109bfad3e2df5aa8e89b837a951bf8ad2cf)

=== 2025-09-25 ===
* 21:20 wmbot~damian@tools-bastion-15: bot deployed @ v1.2.2
* 21:20 wmbot~damian-scripts@tools-bastion-15: bot deployed @ refs/tags/v1.2.2
* 19:17 wmbot~damian-scripts@tools-bastion-15: bot deployed @ refs/tags/v1.2.1
* 19:16 wmbot~damian-scripts@tools-bastion-15: bot deployed @ refs/tags/v1.2.0
* 18:29 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18017003184 (https://github.com/cluebotng/component-configs/commits/e9fc8d46ac1a0ff0ac6203458fa171c6430492ce)
* 18:25 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18016894528 (https://github.com/cluebotng/component-configs/commits/cfc9adc9516df0f11c8b6d1df68232d0a46cb4eb)
* 17:46 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18015998407 (https://github.com/cluebotng/component-configs/commits/5592cdfcdc7e683a993c8e784d83fb1a71a0b04c)
* 17:32 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18015392766 (https://github.com/cluebotng/component-configs/commits/61a7ceac210077c3d81bc064c37f8d8668cc2cfb)
* 17:20 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18015392766 (https://github.com/cluebotng/component-configs/commits/61a7ceac210077c3d81bc064c37f8d8668cc2cfb)
* 16:56 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18014801881 (https://github.com/cluebotng/component-configs/commits/4f92189a79e68827f38e9a6a233b20c02529e77c)
* 16:32 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18014221965 (https://github.com/cluebotng/component-configs/commits/b0737b89fc85c164c5a869aff21421ba21af2e4d)
* 16:15 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18013782359 (https://github.com/cluebotng/component-configs/commits/7e1eb9e3c9a52e0dd71cc58dc797183236a1c27e)
* 16:11 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18013677281 (https://github.com/cluebotng/component-configs/commits/371029d320611d8be6103da43ce9e0a91a2f8e1a)
* 16:06 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18013531091 (https://github.com/cluebotng/component-configs/commits/9a6dc9f53f08ea206e75ad75ddddc3429e1e004f)

=== 2025-09-22 ===
* 19:08 wmbot~damian-scripts@tools-bastion-15: report deployed @ refs/tags/1.2.3
* 19:04 wmbot~damian-scripts@tools-bastion-15: report deployed @ refs/tags/1.2.2
* 19:02 wmbot~damian-scripts@tools-bastion-15: report deployed @ refs/tags/1.2.1

=== 2025-08-29 ===
* 14:30 wmbot~damian-scripts@tools-bastion-13: report deployed @ refs/tags/v1.2.0
* 14:30 wmbot~damian-scripts@tools-bastion-13: report deployed @ refs/heads/main
* 00:10 wmbot~damian-scripts@tools-bastion-13: report deployed @ refs/tags/v1.1.1

=== 2025-08-15 ===
* 21:14 wmbot~damian-scripts@tools-bastion-13: report deployed @ refs/tags/v1.1.0
* 20:58 wmbot~damian-scripts@tools-bastion-13: report deployed @ refs/tags/1.0.34
* 12:58 wmbot~damian-scripts@tools-bastion-13: report deployed @ refs/tags/v1.0.33
* 00:23 wmbot~damian-scripts@tools-bastion-13: report deployed @ refs/tags/v1.0.32
* 00:08 wmbot~damian-scripts@tools-bastion-13: report deployed @ refs/tags/v1.0.31

=== 2025-08-14 ===
* 16:13 wmbot~damian@tools-bastion-13: irc-relay deployed @ v1.1.12
* 12:47 wmbot~damian@tools-bastion-13: core deployed @ v0.0.2
* 12:43 wmbot~damian@tools-bastion-13: core deployed @ v0.0.2

=== 2025-08-11 ===
* 13:06 wmbot~damian-scripts@tools-bastion-13: bot deployed @ refs/tags/v1.1.3
* 12:37 wmbot~damian-scripts@tools-bastion-13: report deployed @ refs/tags/v1.0.30

=== 2025-08-10 ===
* 17:42 wmbot~damian-scripts@tools-bastion-13: bot deployed @ refs/tags/v1.1.2
* 17:37 wmbot~damian-scripts@tools-bastion-13: report deployed @ refs/tags/v1.0.29
* 16:35 wmbot~damian-scripts@tools-bastion-13: bot deployed @ refs/tags/v1.1.1

=== 2025-08-08 ===
* 15:37 wmbot~damian@tools-bastion-12: Updated ci-execute-fabric to use dedicated unix account, dropped key from human account
* 15:36 wmbot~damian-scripts@tools-bastion-13: report deployed @ refs/tags/v1.0.28
* 15:33 wmbot~damian@tools-bastion-13: report deployed @ refs/tags/v1.0.28

=== 2025-08-07 ===
* 15:24 wmbot~damian@tools-bastion-13: report deployed @ v1.0.27

=== 2024-03-21 ===
* 10:13 dcaro: fixed .lighttpd.conf file to add port and remove socket

=== 2021-07-23 ===
* 16:40 majavah: stop cbng_relay grid job, still having issues with irc connection - [[phab:T274871|T274871]]

=== 2020-01-13 ===
* 20:53 wm-bot: &lt;root&gt; Restarted webservice to fix broken registration with the front proxy ([[phab:T242538|T242538]])

=== 2019-03-07 ===
* 05:06 bd808: Killed cbng_bot job stuck in deletion state with 4000+ zombie child processes ([[phab:T217817|T217817]])


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>0eu8u8r93nalmth1qxefvgwwzl7asa8</sha1>
    </revision>
  </page>
  <page>
    <title>Nova Resource:Tools.integraality/SAL</title>
    <ns>498</ns>
    <id>443949</id>
    <revision>
      <id>2408828</id>
      <parentid>2408440</parentid>
      <timestamp>2026-05-03T19:41:29Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>wmbot~jeanfred@tools-bastion-15: Deploy 928f398 (Remove obsolete version attribute from docker-compose.yml)</comment>
      <origin>2408828</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="23343" sha1="tpkh3ubl5f7lr9tc8mnut2lydfr6dbm" xml:space="preserve">=== 2026-05-03 ===
* 19:41 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|928f398}} (Remove obsolete version attribute from docker-compose.yml)

=== 2026-05-01 ===
* 18:16 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|01e97d6}} (Extract grouping link strategies into GroupingLink hierarchy), {{Gerrit|359770d}} (Remove pass-through methods for grouping link in GroupingConfiguration), {{Gerrit|54e4a7b}} (Add lang parameter to get_label_for_variable), {{Gerrit|cccd661}} (Simplify SPARQL for mul language in get_label_for_variable), {{Gerrit|548383d}} (Support any label language for grouping links), {{Gerrit|d571596}} (Support QID-based grouping links), {{Gerrit|cfd7c68}} (
* 18:15 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|19b856d}}, {{Gerrit|473fbeb}}, {{Gerrit|0cb0915}}

=== 2026-03-27 ===
* 22:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2bad8b3}} (Catch ModuleNotFoundError when unpickling stale cache entries)

=== 2026-03-24 ===
* 16:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|7883b73}} (Switch from bare imports to relative imports)
* 15:56 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5dbe412}} (Resolve grouping type eagerly in PropertyStatistics constructor)

=== 2026-03-23 ===
* 22:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|1b60d39}} (Handle stale pickle cache entries referencing removed classes)
* 22:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|124f4ca}} (Replace wikibase:isSomeValue with STRSTARTS filter for QLever compatibility)
* 21:28 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fd5828f}} (Detect grouping type via SPARQL and flatten to single GroupingConfiguration)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|e9fae82}} (Add AbstractGroupingType base class for grouping type strategies)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fee0ed5}} (Extract GroupingType strategy classes from GroupingConfiguration hierarchy)

=== 2026-03-18 ===
* 20:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f9dfb79}} (Add /healthz endpoint for automatic health check)

=== 2026-03-17 ===
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2940962}} (Replace GROUP_MAPPING Enum with SPECIAL_GROUPINGS tuple)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|420658b}} (Add MARKER attribute to special grouping classes)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|976599f}} (Rename UnknownValueGrouping.TITLE to MARKER)

=== 2026-03-14 ===
* 16:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be09048}} (Add Wikimedia Commons support using QLever endpoint) for [[phab:T294893|T294893]]
* 16:38 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|ae53e61}} (Derive QLever UI URL from engine endpoint)
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5134f77}} (Use ka/Ma unit suffixes for geological-scale year grouping headings) for [[phab:T236590|T236590]]
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8a735a0}} (Add automatic date resolution rebinning for year groupings) for [[phab:T236590|T236590]]
* 15:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|6478051}}, {{Gerrit|4206c09}}, {{Gerrit|0e920fe}}, {{Gerrit|4eefea1}}

=== 2026-03-13 ===
* 20:33 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a39f1d9}} (Add proper handling for transient Wikidata server errors) for [[phab:T415439|T415439]]
* 20:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|0c418a5}} (Fix tests hitting live Wikidata API (and potential rate-limits)) for [[phab:T420054|T420054]]

=== 2026-03-11 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9665d8f}} (Remove obsolete GroupingType enum from column.py)
* 21:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|60d500a}} (Remove backward-compatible formatting wrapper methods from PropertyStatistics)
* 21:20 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a452c28}} (Extract formatting logic into separate ResultsFormatter class)
* 17:29 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|02338eb}} (Move heading strings to class constants in Grouping objects) and {{Gerrit|d730307}} (Format NoGroup and Totals in  with the other groupings)

=== 2026-03-09 ===
* 18:32 wmbot~jeanfred@tools-bastion-15: Recreated Python 3.11 virtual environment using 'toolforge webservice python3.11 shell' / 'webservice-python-bootstrap --fresh' / '/data/project/integraality/www/python/venv/bin/pip install -r /data/project/integraality/integraality/requirements.txt'
* 18:30 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|55f1132}} (Switch to Python 3.11 as default interpreter)

=== 2026-03-08 ===
* 11:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|37fcc25}} (Extract jobs-framework runtime image as its own variable) and {{Gerrit|1f9bfbb}} (Add service.template deployment to Ansible)

=== 2026-03-06 ===
* 10:08 wmbot~jeanfred@tools-bastion-15: Successfully installed flask-3.1.3 mwparserfromhell-0.7.2 pywikibot-10.7.6 redis-7.0.1 (using a webservice python3.9 shell)
* 10:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fa536f}}, {{Gerrit|9e5111c}}, {{Gerrit|e176f18}}, {{Gerrit|eff3045}}, {{Gerrit|8532e32}} (Dependencies upgrades)

=== 2026-03-05 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5075934}} (Add support for user-specified groupings via template parameter) for [[phab:T419059|T419059]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|3c9fa98}} (Remove two debug print statements)
* 18:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28d800e}} (Extract rdfs:label SPARQL fragment from grouping.py to get_label_for_variable utility), {{Gerrit|40c39de}} (Make intermediate label variables unique to avoid clashes in get_label_for_variable) and {{Gerrit|64e8f94}} (Replace SERVICE wikibase:label with native SPARQL for QLever compatibility) for [[phab:T385749|T385749]]

=== 2025-11-03 ===
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]

=== 2025-10-29 ===
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f40974f}} (Add links to QLever in the Queries) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28c2e1b}} (Add link to QLever on queries error page) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4cc2bcc}} (Add configurable SPARQL endpoint support for QLever in PagesProcessor) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be5c234}} (Add support for QLever as alternative SPARQL query engine) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f34624d}} (Move pywikibot exception handling to SPARQL engine class) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4da9f33}} (Implement SPARQL query engine dependency injection) for [[phab:T385749|T385749]]

=== 2025-10-28 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|92ab962}} (Update pywikibot from 10.0.0 to latest 10.6.0)
* 21:34 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|77deb17}} (Remove link to 'SGE-jobs' tool in footer)

=== 2025-10-27 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8db01c6}} (Simplify Flask webserver startup in app.py)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|31f14b0}} (Bump ruff pre-commit hook from 0.9.3 to latest 0.14.2)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|995121d}} (Bump uv pre-commit hook from 0.5.24 to latest 0.9.5 and regenerate requirements files)
* 09:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|69669c0}} (Optimize 'get_grouping_information_query' query using subquery) for [[phab:T400480|T400480]]

=== 2025-10-25 ===
* 09:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9ac4c1f}} (Convert SPARQL query tests to use triple-quoted strings in PropertyStatisticsTest)

=== 2025-10-24 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5511fd0}} (Fix test_get_grouping_information_query_with_grouping_link for YearGrouping)
* 19:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8bb5d37}} (Convert SPARQL query tests to use triple-quoted strings)
* 19:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|792b38e}} (Remove redundant GROUP BY parameter in 'get_grouping_information_query')

=== 2025-05-06 ===
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|284a70a}} (Update pywikibot from 9.6.1 to latest 10.0.0)
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|1f030a9}} (Re-generate uv lockfile to add 'upload-time' markers)
* 11:19 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8f2391e}} (Replace deprecated  with  in two unit tests)

=== 2025-05-04 ===
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7a2fa93}} (Fallback to mul when formatting grouping links) for [[phab:T393166|T393166]]
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)

=== 2025-05-03 ===
* 19:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|781934d}} (Move grouping-link to Grouping objects and retrieve via SPARQL) for [[phab:T237276|T237276]]

=== 2025-05-02 ===
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5bc6fe9}} (Remove unused local variable in GroupingTest.test unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|f5f0044}} (Remove unused local variable in  unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a742a2}} (Remove unused column_key local variable from get_query_for_items_for_property_[posi{{!}}nega]tive)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|620ac00}} (Remove unused patcher in QueriesTests setUp method)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|e32b9b9}} (Allow to optionally pass wikiproject data to SitelinkColumn constructor)

=== 2025-05-01 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|2a91226}} (Catch WDQS Timeouts and raise proper exception for all SPARQL queries) for [[phab:T278156|T278156]]
* 20:29 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|397c5e8}} (Also catch ServerError to handle WDQS timeouts) for [[phab:T384882|T384882]]

=== 2025-04-28 ===
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|eba498e}} (Use correct namespace when using pywikibot exceptions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ea55660}} (Add unit-tests for page_saving.py functions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|58918e6}} (Tweak linebreak in get_grouping_information_query which was breaking unit tests)

=== 2025-01-23 ===
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|a76ceec}} (Switch from Pipenv to uv as project manager)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5941f3f}} (Upgrade all Python dev-dependencies)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix 'QueriesTests' unit tests in test_app)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix  unit tests in test_app)

=== 2025-01-17 ===
* 10:03 wmbot~jeanfred@tools-sgebastion-10: Recreated virtual environment using 'toolforge webservice python3.9 shell' / 'webservice-python-bootstrap --fresh'

=== 2025-01-16 ===
* 16:36 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|525c9ed}} (Upgrade all Python dependencies)

=== 2024-07-01 ===
* 19:54 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8eecfc6}} &amp; {{Gerrit|0466207}}

=== 2024-06-06 ===
* 12:47 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|022fa2f}} ([[phab:T312727|T312727]])
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ec28a13}}
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7adc07f}}

=== 2024-05-10 ===
* 15:57 wmbot~multichill@tools-bastion-12: Leaving project

=== 2024-05-04 ===
* 20:22 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|51ba697}}
* 20:07 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8ec2fec}} ([[phab:T251008|T251008]])

=== 2024-05-03 ===
* 12:01 wmbot~multichill@tools-bastion-12: emails: all -

=== 2024-04-30 ===
* 19:10 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5a91a78}}, {{Gerrit|864d518}}, c9b199 ([[phab:T251008|T251008]]), {{Gerrit|0dd731f}}

=== 2024-03-17 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a93a59}} ([[phab:T319813|T319813]])
* 20:25 wmbot~jeanfred@tools-sgebastion-10: Re-created virtual environment for Python 3.9 ([[phab:T319813|T319813]])
* 19:26 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|bde5715}}, {{Gerrit|9bdfb02}} ([[phab:T319813|T319813]])

=== 2023-11-21 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|992670c}} ([[phab:T351574|T351574]])
* 19:55 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|1f41a79}}, {{Gerrit|38e7623}}

=== 2023-10-17 ===
* 14:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d7f09b3}}, {{Gerrit|eb1de62}}, {{Gerrit|f4e3656}}, {{Gerrit|82f90c4}} ([[phab:T312726|T312726]])

=== 2023-10-03 ===
* 21:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 19:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 18:04 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|da99820}}, {{Gerrit|801ddeb}}, {{Gerrit|f2ffd22}} ([[phab:T312729|T312729]])

=== 2023-10-02 ===
* 09:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3f9a3d5}}

=== 2023-09-26 ===
* 18:25 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3caa9ed}}, {{Gerrit|a8029ed}}, {{Gerrit|c88ad35}}, {{Gerrit|e575508}}, {{Gerrit|3e3f47f}}, {{Gerrit|63ab297}}, {{Gerrit|06cb52d}}, {{Gerrit|ac64840}}, {{Gerrit|6178698}} ([[phab:T312728|T312728]])

=== 2023-06-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5aecae8}}, {{Gerrit|649259f}}, {{Gerrit|9712666}} ([[phab:T338684|T338684]])

=== 2022-12-26 ===
* 10:48 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c90f9ef}} ([[phab:T325936|T325936]])

=== 2022-09-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|bcbd306}}

=== 2022-07-12 ===
* 15:04 wm-bot: &lt;jeanfred&gt; Triggering full update
* 15:02 wm-bot: &lt;jeanfred&gt; jstop {{Gerrit|5976623}} to kill weekly update job stuck since May 22

=== 2022-06-03 ===
* 12:37 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9e85ede}} ([[phab:T309861|T309861]])

=== 2022-05-15 ===
* 21:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3a675b6}}
* 20:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|80f073e}}
* 19:31 wm-bot: &lt;jeanfred&gt; Uninstall removed requirements and restart webservice
* 19:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|f6dc34b}}
* 19:15 wm-bot: &lt;jeanfred&gt; Install all requirements and restart webservice
* 19:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d395b4c}}, {{Gerrit|591c450}}, {{Gerrit|6d09e5b}}

=== 2022-05-14 ===
* 14:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2a521d1}}, {{Gerrit|29d6b62}}, {{Gerrit|8bd793d}} ([[phab:T257942|T257942]])

=== 2022-05-11 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|43f07d2}}, {{Gerrit|da1f2ef}}, {{Gerrit|6ab7547}}, {{Gerrit|5f6b20a}}, {{Gerrit|0df2121}}

=== 2022-01-28 ===
* 20:44 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|13ed7a3}}

=== 2021-11-01 ===
* 20:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|972a1df}}

=== 2021-10-30 ===
* 19:36 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c1628b9}} and {{Gerrit|b6940b0}}

=== 2021-10-29 ===
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|40c8cd3}}
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|af49bab}} ([[phab:T236590|T236590]] and [[phab:T273226|T273226]])
* 09:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2260c8f}} (follow up to [[phab:T294570|T294570]])
* 09:16 wm-bot: &lt;jeanfred&gt; Install requirements in py37 venv ([[phab:T294570|T294570]])
* 09:14 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a4560a}} and {{Gerrit|de296cd}} ([[phab:T294570|T294570]])

=== 2021-10-28 ===
* 17:14 wm-bot: &lt;jeanfred&gt; Rollbacked to {{Gerrit|977236b42daef66818e561b49e64264a0c785454}} for [[phab:T294570|T294570]] and restart
* 12:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a8e7fb}} ([[phab:T236590|T236590]])
* 11:31 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|30a2f2ba}} ([[phab:T278156|T278156]])
* 11:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5c547d8}}
* 10:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b4f12bc}} to help investigate [[phab:T236960|T236960]]
* 09:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|137f5b8}}, {{Gerrit|c816e41}} {{Gerrit|8e0b9ca}}, {{Gerrit|8dbc099}} ([[phab:T273226|T273226]])

=== 2021-10-27 ===
* 09:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6194d84}} ([[phab:T273226|T273226]])

=== 2021-10-25 ===
* 21:41 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|7e3343a}} ([[phab:T278156|T278156]])
* 20:22 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|ef0f537}} ([[phab:T284684|T284684]])
* 20:15 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|977236b}} ([[phab:T284183|T284183]])
* 18:47 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5dfef39}}, {{Gerrit|e940ae7}}, {{Gerrit|765b314}}, {{Gerrit|3e4c2de}}, {{Gerrit|c859c45}}, {{Gerrit|5526cb3}}

=== 2021-06-11 ===
* 15:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|a753cb2}}
* 15:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5187a4d}}

=== 2021-05-31 ===
* 09:20 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|79e458d}}

=== 2021-05-22 ===
* 20:58 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|258aae9a}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|67c42f43}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d170ea1}}
* 20:24 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2f39621}}
* 20:21 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b52079f}}
* 17:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|8ffbad82c3}} ([[phab:T279236|T279236]])
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6de2308}} and {{Gerrit|c436760}} ([[phab:T279236|T279236]])
* 16:06 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0966be04}} ([[phab:T279236|T279236]])
* 15:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9727a01c}}
* 14:02 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|69048ef}}
* 13:00 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|633eeb0f}}
* 12:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d4fa363}} ([[phab:T283287|T283287]])

=== 2021-04-03 ===
* 21:34 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|666222b}} and {{Gerrit|0dcafff}} ([[phab:T279236|T279236]])
* 21:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])
* 16:45 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])

=== 2021-03-26 ===
* 20:57 wm-bot: &lt;jeanfred&gt; Reinstalled requirements in both virtual environments for [[phab:T240312|T240312]]
* 20:55 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|bcc4fe41}} ([[phab:T240312|T240312]])

=== 2020-07-06 ===
* 22:55 wm-bot: &lt;root&gt; Migrated .webservicerc to service.template ([[phab:T257229|T257229]])

=== 2020-06-18 ===
* 11:01 wm-bot: &lt;jeanfred&gt; Restarted with --canonical for Toolforge domain migration

=== 2020-05-01 ===
* 22:59 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|f0db935}}, {{Gerrit|f8d9fdf}}, {{Gerrit|f57c060}}, {{Gerrit|6c534b7}} ([[phab:T248788|T248788]])

=== 2020-04-29 ===
* 20:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eabbac8}}, {{Gerrit|b74b352}} ([[phab:T248788|T248788]])

=== 2020-04-24 ===
* 22:22 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6963040}}, {{Gerrit|35a3ea7}}, {{Gerrit|22e27b5}}, {{Gerrit|9dd44ff}}, {{Gerrit|0e45815}}, {{Gerrit|6f36ffd}} ([[phab:T248788|T248788]])
* 19:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|9fea19b}}, {{Gerrit|f38047e}}

=== 2020-02-25 ===
* 23:35 wm-bot: &lt;root&gt; Migrated to 2020 Kubernetes cluster

=== 2020-02-16 ===
* 16:49 JeanFred: &quot;Deploy latest from Git master: {{Gerrit|0fcb522}} ([[phab:T243780|T243780]]), {{Gerrit|4cb6a5a}} ([[phab:T243780|T243780]], [[phab:T239067|T239067]])&quot;

=== 2020-02-11 ===
* 21:23 wm-bot157: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|c350e4f}} ([[phab:T243998|T243998]]), {{Gerrit|fa12c21}}

=== 2020-02-05 ===
* 12:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8977958}} ([[phab:T243780|T243780]])

=== 2020-02-03 ===
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|3b2867f}}, {{Gerrit|10f67fd}}, {{Gerrit|2e6a669}} ([[phab:T244030|T244030]]), {{Gerrit|279c92b}}, {{Gerrit|de6308e}}

=== 2020-01-16 ===
* 21:44 wm-bot: &lt;jeanfred&gt; Perform webservice restart for [[phab:T242967|T242967]]

=== 2019-12-11 ===
* 15:15 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|257618f1}} ([[phab:T240312|T240312]])

=== 2019-12-02 ===
* 06:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|770cc93}} ([[phab:T237182|T237182]])

=== 2019-11-25 ===
* 11:36 wm-bot: &lt;jeanfred&gt; Trigger a manual update of all dashboards ([[phab:T239085|T239085]])
* 11:34 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|684fb77}} ([[phab:T239085|T239085]])

=== 2019-11-21 ===
* 08:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2f94256}} ([[phab:T237187|T237187]])

=== 2019-11-10 ===
* 15:31 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|ab714de}} ([[phab:T224226|T224226]])

=== 2019-11-06 ===
* 09:12 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|7202182}} ([[phab:T237187|T237187]])
* 09:01 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|29b19c3}} ([[phab:T237182|T237182]])

=== 2019-11-04 ===
* 17:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d0f4222}} ([[phab:T237189|T237189]])
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|588dae8}}
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|471deddd}} ([[phab:T237182|T237182]])

=== 2019-10-31 ===
* 09:06 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|46484f5}} ([[phab:T224226|T224226]])
* 09:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2d71154}}, {{Gerrit|d0d2937}}, {{Gerrit|905007b}}, {{Gerrit|effee36}}, {{Gerrit|7c15083}}

=== 2019-10-30 ===
* 19:42 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|289a41b}}, {{Gerrit|02a1a4f}}, {{Gerrit|e8d4363}} ([[phab:T224226|T224226]])

=== 2019-10-25 ===
* 15:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|82de2957}} ([[phab:T224212|T224212]], [[phab:T223930|T223930]])
* 14:08 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6a055c2}} ([[phab:T228405|T228405]])
* 13:24 wm-bot: &lt;jeanfred&gt; Deploying from master, will break things.

=== 2019-09-19 ===
* 18:38 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d214f09}}, {{Gerrit|9b8dfa5}}
* 17:27 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|12ebc39}}

=== 2019-07-07 ===
* 22:49 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|5b4a1c6}}, {{Gerrit|ab90227}}, {{Gerrit|d101ca1}}, {{Gerrit|394be6a}}, {{Gerrit|ae16d68}}, {{Gerrit|728a8d2}}

=== 2019-05-30 ===
* 12:08 wm-bot: &lt;jeanfred&gt; Nuked the virtualenv and reinstalled all deps from scratch, in desperation for [[phab:T224651|T224651]]
* 10:59 wm-bot: &lt;jeanfred&gt; Service stop, mv logs, service start for [[phab:T224651|T224651]]

=== 2019-05-22 ===
* 16:39 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8e76a8a}}
* 13:13 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2fdede2}}, {{Gerrit|6b6c07c}}, {{Gerrit|c17cb78}} ([[phab:T224090|T224090]])

=== 2019-05-20 ===
* 19:01 wm-bot: &lt;jeanfred&gt; Restart uwsgi with 10 workers rather than the default of 4
* 19:00 wm-bot: &lt;jeanfred&gt; --help


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>tpkh3ubl5f7lr9tc8mnut2lydfr6dbm</sha1>
    </revision>
    <revision>
      <id>2408829</id>
      <parentid>2408828</parentid>
      <timestamp>2026-05-03T19:44:18Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>wmbot~jeanfred@tools-bastion-15: Deploy 4dcc1a0 (Extract prepare_report_groupings from process_data in PropertyStatistics) for T368001</comment>
      <origin>2408829</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="23514" sha1="tolxg375ii4cth4or667ylc5q3yt255" xml:space="preserve">=== 2026-05-03 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4dcc1a0}} (Extract prepare_report_groupings from process_data in PropertyStatistics) for [[phab:T368001|T368001]]
* 19:41 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|928f398}} (Remove obsolete version attribute from docker-compose.yml)

=== 2026-05-01 ===
* 18:16 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|01e97d6}} (Extract grouping link strategies into GroupingLink hierarchy), {{Gerrit|359770d}} (Remove pass-through methods for grouping link in GroupingConfiguration), {{Gerrit|54e4a7b}} (Add lang parameter to get_label_for_variable), {{Gerrit|cccd661}} (Simplify SPARQL for mul language in get_label_for_variable), {{Gerrit|548383d}} (Support any label language for grouping links), {{Gerrit|d571596}} (Support QID-based grouping links), {{Gerrit|cfd7c68}} (
* 18:15 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|19b856d}}, {{Gerrit|473fbeb}}, {{Gerrit|0cb0915}}

=== 2026-03-27 ===
* 22:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2bad8b3}} (Catch ModuleNotFoundError when unpickling stale cache entries)

=== 2026-03-24 ===
* 16:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|7883b73}} (Switch from bare imports to relative imports)
* 15:56 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5dbe412}} (Resolve grouping type eagerly in PropertyStatistics constructor)

=== 2026-03-23 ===
* 22:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|1b60d39}} (Handle stale pickle cache entries referencing removed classes)
* 22:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|124f4ca}} (Replace wikibase:isSomeValue with STRSTARTS filter for QLever compatibility)
* 21:28 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fd5828f}} (Detect grouping type via SPARQL and flatten to single GroupingConfiguration)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|e9fae82}} (Add AbstractGroupingType base class for grouping type strategies)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fee0ed5}} (Extract GroupingType strategy classes from GroupingConfiguration hierarchy)

=== 2026-03-18 ===
* 20:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f9dfb79}} (Add /healthz endpoint for automatic health check)

=== 2026-03-17 ===
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2940962}} (Replace GROUP_MAPPING Enum with SPECIAL_GROUPINGS tuple)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|420658b}} (Add MARKER attribute to special grouping classes)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|976599f}} (Rename UnknownValueGrouping.TITLE to MARKER)

=== 2026-03-14 ===
* 16:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be09048}} (Add Wikimedia Commons support using QLever endpoint) for [[phab:T294893|T294893]]
* 16:38 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|ae53e61}} (Derive QLever UI URL from engine endpoint)
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5134f77}} (Use ka/Ma unit suffixes for geological-scale year grouping headings) for [[phab:T236590|T236590]]
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8a735a0}} (Add automatic date resolution rebinning for year groupings) for [[phab:T236590|T236590]]
* 15:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|6478051}}, {{Gerrit|4206c09}}, {{Gerrit|0e920fe}}, {{Gerrit|4eefea1}}

=== 2026-03-13 ===
* 20:33 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a39f1d9}} (Add proper handling for transient Wikidata server errors) for [[phab:T415439|T415439]]
* 20:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|0c418a5}} (Fix tests hitting live Wikidata API (and potential rate-limits)) for [[phab:T420054|T420054]]

=== 2026-03-11 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9665d8f}} (Remove obsolete GroupingType enum from column.py)
* 21:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|60d500a}} (Remove backward-compatible formatting wrapper methods from PropertyStatistics)
* 21:20 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a452c28}} (Extract formatting logic into separate ResultsFormatter class)
* 17:29 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|02338eb}} (Move heading strings to class constants in Grouping objects) and {{Gerrit|d730307}} (Format NoGroup and Totals in  with the other groupings)

=== 2026-03-09 ===
* 18:32 wmbot~jeanfred@tools-bastion-15: Recreated Python 3.11 virtual environment using 'toolforge webservice python3.11 shell' / 'webservice-python-bootstrap --fresh' / '/data/project/integraality/www/python/venv/bin/pip install -r /data/project/integraality/integraality/requirements.txt'
* 18:30 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|55f1132}} (Switch to Python 3.11 as default interpreter)

=== 2026-03-08 ===
* 11:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|37fcc25}} (Extract jobs-framework runtime image as its own variable) and {{Gerrit|1f9bfbb}} (Add service.template deployment to Ansible)

=== 2026-03-06 ===
* 10:08 wmbot~jeanfred@tools-bastion-15: Successfully installed flask-3.1.3 mwparserfromhell-0.7.2 pywikibot-10.7.6 redis-7.0.1 (using a webservice python3.9 shell)
* 10:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fa536f}}, {{Gerrit|9e5111c}}, {{Gerrit|e176f18}}, {{Gerrit|eff3045}}, {{Gerrit|8532e32}} (Dependencies upgrades)

=== 2026-03-05 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5075934}} (Add support for user-specified groupings via template parameter) for [[phab:T419059|T419059]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|3c9fa98}} (Remove two debug print statements)
* 18:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28d800e}} (Extract rdfs:label SPARQL fragment from grouping.py to get_label_for_variable utility), {{Gerrit|40c39de}} (Make intermediate label variables unique to avoid clashes in get_label_for_variable) and {{Gerrit|64e8f94}} (Replace SERVICE wikibase:label with native SPARQL for QLever compatibility) for [[phab:T385749|T385749]]

=== 2025-11-03 ===
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]

=== 2025-10-29 ===
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f40974f}} (Add links to QLever in the Queries) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28c2e1b}} (Add link to QLever on queries error page) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4cc2bcc}} (Add configurable SPARQL endpoint support for QLever in PagesProcessor) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be5c234}} (Add support for QLever as alternative SPARQL query engine) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f34624d}} (Move pywikibot exception handling to SPARQL engine class) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4da9f33}} (Implement SPARQL query engine dependency injection) for [[phab:T385749|T385749]]

=== 2025-10-28 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|92ab962}} (Update pywikibot from 10.0.0 to latest 10.6.0)
* 21:34 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|77deb17}} (Remove link to 'SGE-jobs' tool in footer)

=== 2025-10-27 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8db01c6}} (Simplify Flask webserver startup in app.py)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|31f14b0}} (Bump ruff pre-commit hook from 0.9.3 to latest 0.14.2)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|995121d}} (Bump uv pre-commit hook from 0.5.24 to latest 0.9.5 and regenerate requirements files)
* 09:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|69669c0}} (Optimize 'get_grouping_information_query' query using subquery) for [[phab:T400480|T400480]]

=== 2025-10-25 ===
* 09:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9ac4c1f}} (Convert SPARQL query tests to use triple-quoted strings in PropertyStatisticsTest)

=== 2025-10-24 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5511fd0}} (Fix test_get_grouping_information_query_with_grouping_link for YearGrouping)
* 19:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8bb5d37}} (Convert SPARQL query tests to use triple-quoted strings)
* 19:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|792b38e}} (Remove redundant GROUP BY parameter in 'get_grouping_information_query')

=== 2025-05-06 ===
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|284a70a}} (Update pywikibot from 9.6.1 to latest 10.0.0)
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|1f030a9}} (Re-generate uv lockfile to add 'upload-time' markers)
* 11:19 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8f2391e}} (Replace deprecated  with  in two unit tests)

=== 2025-05-04 ===
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7a2fa93}} (Fallback to mul when formatting grouping links) for [[phab:T393166|T393166]]
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)

=== 2025-05-03 ===
* 19:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|781934d}} (Move grouping-link to Grouping objects and retrieve via SPARQL) for [[phab:T237276|T237276]]

=== 2025-05-02 ===
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5bc6fe9}} (Remove unused local variable in GroupingTest.test unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|f5f0044}} (Remove unused local variable in  unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a742a2}} (Remove unused column_key local variable from get_query_for_items_for_property_[posi{{!}}nega]tive)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|620ac00}} (Remove unused patcher in QueriesTests setUp method)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|e32b9b9}} (Allow to optionally pass wikiproject data to SitelinkColumn constructor)

=== 2025-05-01 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|2a91226}} (Catch WDQS Timeouts and raise proper exception for all SPARQL queries) for [[phab:T278156|T278156]]
* 20:29 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|397c5e8}} (Also catch ServerError to handle WDQS timeouts) for [[phab:T384882|T384882]]

=== 2025-04-28 ===
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|eba498e}} (Use correct namespace when using pywikibot exceptions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ea55660}} (Add unit-tests for page_saving.py functions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|58918e6}} (Tweak linebreak in get_grouping_information_query which was breaking unit tests)

=== 2025-01-23 ===
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|a76ceec}} (Switch from Pipenv to uv as project manager)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5941f3f}} (Upgrade all Python dev-dependencies)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix 'QueriesTests' unit tests in test_app)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix  unit tests in test_app)

=== 2025-01-17 ===
* 10:03 wmbot~jeanfred@tools-sgebastion-10: Recreated virtual environment using 'toolforge webservice python3.9 shell' / 'webservice-python-bootstrap --fresh'

=== 2025-01-16 ===
* 16:36 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|525c9ed}} (Upgrade all Python dependencies)

=== 2024-07-01 ===
* 19:54 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8eecfc6}} &amp; {{Gerrit|0466207}}

=== 2024-06-06 ===
* 12:47 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|022fa2f}} ([[phab:T312727|T312727]])
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ec28a13}}
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7adc07f}}

=== 2024-05-10 ===
* 15:57 wmbot~multichill@tools-bastion-12: Leaving project

=== 2024-05-04 ===
* 20:22 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|51ba697}}
* 20:07 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8ec2fec}} ([[phab:T251008|T251008]])

=== 2024-05-03 ===
* 12:01 wmbot~multichill@tools-bastion-12: emails: all -

=== 2024-04-30 ===
* 19:10 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5a91a78}}, {{Gerrit|864d518}}, c9b199 ([[phab:T251008|T251008]]), {{Gerrit|0dd731f}}

=== 2024-03-17 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a93a59}} ([[phab:T319813|T319813]])
* 20:25 wmbot~jeanfred@tools-sgebastion-10: Re-created virtual environment for Python 3.9 ([[phab:T319813|T319813]])
* 19:26 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|bde5715}}, {{Gerrit|9bdfb02}} ([[phab:T319813|T319813]])

=== 2023-11-21 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|992670c}} ([[phab:T351574|T351574]])
* 19:55 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|1f41a79}}, {{Gerrit|38e7623}}

=== 2023-10-17 ===
* 14:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d7f09b3}}, {{Gerrit|eb1de62}}, {{Gerrit|f4e3656}}, {{Gerrit|82f90c4}} ([[phab:T312726|T312726]])

=== 2023-10-03 ===
* 21:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 19:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 18:04 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|da99820}}, {{Gerrit|801ddeb}}, {{Gerrit|f2ffd22}} ([[phab:T312729|T312729]])

=== 2023-10-02 ===
* 09:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3f9a3d5}}

=== 2023-09-26 ===
* 18:25 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3caa9ed}}, {{Gerrit|a8029ed}}, {{Gerrit|c88ad35}}, {{Gerrit|e575508}}, {{Gerrit|3e3f47f}}, {{Gerrit|63ab297}}, {{Gerrit|06cb52d}}, {{Gerrit|ac64840}}, {{Gerrit|6178698}} ([[phab:T312728|T312728]])

=== 2023-06-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5aecae8}}, {{Gerrit|649259f}}, {{Gerrit|9712666}} ([[phab:T338684|T338684]])

=== 2022-12-26 ===
* 10:48 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c90f9ef}} ([[phab:T325936|T325936]])

=== 2022-09-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|bcbd306}}

=== 2022-07-12 ===
* 15:04 wm-bot: &lt;jeanfred&gt; Triggering full update
* 15:02 wm-bot: &lt;jeanfred&gt; jstop {{Gerrit|5976623}} to kill weekly update job stuck since May 22

=== 2022-06-03 ===
* 12:37 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9e85ede}} ([[phab:T309861|T309861]])

=== 2022-05-15 ===
* 21:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3a675b6}}
* 20:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|80f073e}}
* 19:31 wm-bot: &lt;jeanfred&gt; Uninstall removed requirements and restart webservice
* 19:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|f6dc34b}}
* 19:15 wm-bot: &lt;jeanfred&gt; Install all requirements and restart webservice
* 19:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d395b4c}}, {{Gerrit|591c450}}, {{Gerrit|6d09e5b}}

=== 2022-05-14 ===
* 14:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2a521d1}}, {{Gerrit|29d6b62}}, {{Gerrit|8bd793d}} ([[phab:T257942|T257942]])

=== 2022-05-11 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|43f07d2}}, {{Gerrit|da1f2ef}}, {{Gerrit|6ab7547}}, {{Gerrit|5f6b20a}}, {{Gerrit|0df2121}}

=== 2022-01-28 ===
* 20:44 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|13ed7a3}}

=== 2021-11-01 ===
* 20:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|972a1df}}

=== 2021-10-30 ===
* 19:36 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c1628b9}} and {{Gerrit|b6940b0}}

=== 2021-10-29 ===
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|40c8cd3}}
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|af49bab}} ([[phab:T236590|T236590]] and [[phab:T273226|T273226]])
* 09:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2260c8f}} (follow up to [[phab:T294570|T294570]])
* 09:16 wm-bot: &lt;jeanfred&gt; Install requirements in py37 venv ([[phab:T294570|T294570]])
* 09:14 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a4560a}} and {{Gerrit|de296cd}} ([[phab:T294570|T294570]])

=== 2021-10-28 ===
* 17:14 wm-bot: &lt;jeanfred&gt; Rollbacked to {{Gerrit|977236b42daef66818e561b49e64264a0c785454}} for [[phab:T294570|T294570]] and restart
* 12:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a8e7fb}} ([[phab:T236590|T236590]])
* 11:31 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|30a2f2ba}} ([[phab:T278156|T278156]])
* 11:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5c547d8}}
* 10:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b4f12bc}} to help investigate [[phab:T236960|T236960]]
* 09:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|137f5b8}}, {{Gerrit|c816e41}} {{Gerrit|8e0b9ca}}, {{Gerrit|8dbc099}} ([[phab:T273226|T273226]])

=== 2021-10-27 ===
* 09:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6194d84}} ([[phab:T273226|T273226]])

=== 2021-10-25 ===
* 21:41 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|7e3343a}} ([[phab:T278156|T278156]])
* 20:22 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|ef0f537}} ([[phab:T284684|T284684]])
* 20:15 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|977236b}} ([[phab:T284183|T284183]])
* 18:47 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5dfef39}}, {{Gerrit|e940ae7}}, {{Gerrit|765b314}}, {{Gerrit|3e4c2de}}, {{Gerrit|c859c45}}, {{Gerrit|5526cb3}}

=== 2021-06-11 ===
* 15:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|a753cb2}}
* 15:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5187a4d}}

=== 2021-05-31 ===
* 09:20 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|79e458d}}

=== 2021-05-22 ===
* 20:58 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|258aae9a}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|67c42f43}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d170ea1}}
* 20:24 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2f39621}}
* 20:21 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b52079f}}
* 17:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|8ffbad82c3}} ([[phab:T279236|T279236]])
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6de2308}} and {{Gerrit|c436760}} ([[phab:T279236|T279236]])
* 16:06 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0966be04}} ([[phab:T279236|T279236]])
* 15:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9727a01c}}
* 14:02 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|69048ef}}
* 13:00 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|633eeb0f}}
* 12:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d4fa363}} ([[phab:T283287|T283287]])

=== 2021-04-03 ===
* 21:34 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|666222b}} and {{Gerrit|0dcafff}} ([[phab:T279236|T279236]])
* 21:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])
* 16:45 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])

=== 2021-03-26 ===
* 20:57 wm-bot: &lt;jeanfred&gt; Reinstalled requirements in both virtual environments for [[phab:T240312|T240312]]
* 20:55 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|bcc4fe41}} ([[phab:T240312|T240312]])

=== 2020-07-06 ===
* 22:55 wm-bot: &lt;root&gt; Migrated .webservicerc to service.template ([[phab:T257229|T257229]])

=== 2020-06-18 ===
* 11:01 wm-bot: &lt;jeanfred&gt; Restarted with --canonical for Toolforge domain migration

=== 2020-05-01 ===
* 22:59 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|f0db935}}, {{Gerrit|f8d9fdf}}, {{Gerrit|f57c060}}, {{Gerrit|6c534b7}} ([[phab:T248788|T248788]])

=== 2020-04-29 ===
* 20:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eabbac8}}, {{Gerrit|b74b352}} ([[phab:T248788|T248788]])

=== 2020-04-24 ===
* 22:22 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6963040}}, {{Gerrit|35a3ea7}}, {{Gerrit|22e27b5}}, {{Gerrit|9dd44ff}}, {{Gerrit|0e45815}}, {{Gerrit|6f36ffd}} ([[phab:T248788|T248788]])
* 19:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|9fea19b}}, {{Gerrit|f38047e}}

=== 2020-02-25 ===
* 23:35 wm-bot: &lt;root&gt; Migrated to 2020 Kubernetes cluster

=== 2020-02-16 ===
* 16:49 JeanFred: &quot;Deploy latest from Git master: {{Gerrit|0fcb522}} ([[phab:T243780|T243780]]), {{Gerrit|4cb6a5a}} ([[phab:T243780|T243780]], [[phab:T239067|T239067]])&quot;

=== 2020-02-11 ===
* 21:23 wm-bot157: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|c350e4f}} ([[phab:T243998|T243998]]), {{Gerrit|fa12c21}}

=== 2020-02-05 ===
* 12:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8977958}} ([[phab:T243780|T243780]])

=== 2020-02-03 ===
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|3b2867f}}, {{Gerrit|10f67fd}}, {{Gerrit|2e6a669}} ([[phab:T244030|T244030]]), {{Gerrit|279c92b}}, {{Gerrit|de6308e}}

=== 2020-01-16 ===
* 21:44 wm-bot: &lt;jeanfred&gt; Perform webservice restart for [[phab:T242967|T242967]]

=== 2019-12-11 ===
* 15:15 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|257618f1}} ([[phab:T240312|T240312]])

=== 2019-12-02 ===
* 06:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|770cc93}} ([[phab:T237182|T237182]])

=== 2019-11-25 ===
* 11:36 wm-bot: &lt;jeanfred&gt; Trigger a manual update of all dashboards ([[phab:T239085|T239085]])
* 11:34 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|684fb77}} ([[phab:T239085|T239085]])

=== 2019-11-21 ===
* 08:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2f94256}} ([[phab:T237187|T237187]])

=== 2019-11-10 ===
* 15:31 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|ab714de}} ([[phab:T224226|T224226]])

=== 2019-11-06 ===
* 09:12 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|7202182}} ([[phab:T237187|T237187]])
* 09:01 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|29b19c3}} ([[phab:T237182|T237182]])

=== 2019-11-04 ===
* 17:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d0f4222}} ([[phab:T237189|T237189]])
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|588dae8}}
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|471deddd}} ([[phab:T237182|T237182]])

=== 2019-10-31 ===
* 09:06 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|46484f5}} ([[phab:T224226|T224226]])
* 09:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2d71154}}, {{Gerrit|d0d2937}}, {{Gerrit|905007b}}, {{Gerrit|effee36}}, {{Gerrit|7c15083}}

=== 2019-10-30 ===
* 19:42 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|289a41b}}, {{Gerrit|02a1a4f}}, {{Gerrit|e8d4363}} ([[phab:T224226|T224226]])

=== 2019-10-25 ===
* 15:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|82de2957}} ([[phab:T224212|T224212]], [[phab:T223930|T223930]])
* 14:08 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6a055c2}} ([[phab:T228405|T228405]])
* 13:24 wm-bot: &lt;jeanfred&gt; Deploying from master, will break things.

=== 2019-09-19 ===
* 18:38 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d214f09}}, {{Gerrit|9b8dfa5}}
* 17:27 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|12ebc39}}

=== 2019-07-07 ===
* 22:49 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|5b4a1c6}}, {{Gerrit|ab90227}}, {{Gerrit|d101ca1}}, {{Gerrit|394be6a}}, {{Gerrit|ae16d68}}, {{Gerrit|728a8d2}}

=== 2019-05-30 ===
* 12:08 wm-bot: &lt;jeanfred&gt; Nuked the virtualenv and reinstalled all deps from scratch, in desperation for [[phab:T224651|T224651]]
* 10:59 wm-bot: &lt;jeanfred&gt; Service stop, mv logs, service start for [[phab:T224651|T224651]]

=== 2019-05-22 ===
* 16:39 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8e76a8a}}
* 13:13 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2fdede2}}, {{Gerrit|6b6c07c}}, {{Gerrit|c17cb78}} ([[phab:T224090|T224090]])

=== 2019-05-20 ===
* 19:01 wm-bot: &lt;jeanfred&gt; Restart uwsgi with 10 workers rather than the default of 4
* 19:00 wm-bot: &lt;jeanfred&gt; --help


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>tolxg375ii4cth4or667ylc5q3yt255</sha1>
    </revision>
    <revision>
      <id>2408830</id>
      <parentid>2408829</parentid>
      <timestamp>2026-05-03T19:44:20Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>wmbot~jeanfred@tools-bastion-15: Deploy 9b32a82 (Split retrieve_and_process_data in process_page) for T368001</comment>
      <origin>2408830</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="23660" sha1="i16hibljdvuu6jqtyegeki3hjn7f50a" xml:space="preserve">=== 2026-05-03 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9b32a82}} (Split retrieve_and_process_data in process_page) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4dcc1a0}} (Extract prepare_report_groupings from process_data in PropertyStatistics) for [[phab:T368001|T368001]]
* 19:41 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|928f398}} (Remove obsolete version attribute from docker-compose.yml)

=== 2026-05-01 ===
* 18:16 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|01e97d6}} (Extract grouping link strategies into GroupingLink hierarchy), {{Gerrit|359770d}} (Remove pass-through methods for grouping link in GroupingConfiguration), {{Gerrit|54e4a7b}} (Add lang parameter to get_label_for_variable), {{Gerrit|cccd661}} (Simplify SPARQL for mul language in get_label_for_variable), {{Gerrit|548383d}} (Support any label language for grouping links), {{Gerrit|d571596}} (Support QID-based grouping links), {{Gerrit|cfd7c68}} (
* 18:15 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|19b856d}}, {{Gerrit|473fbeb}}, {{Gerrit|0cb0915}}

=== 2026-03-27 ===
* 22:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2bad8b3}} (Catch ModuleNotFoundError when unpickling stale cache entries)

=== 2026-03-24 ===
* 16:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|7883b73}} (Switch from bare imports to relative imports)
* 15:56 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5dbe412}} (Resolve grouping type eagerly in PropertyStatistics constructor)

=== 2026-03-23 ===
* 22:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|1b60d39}} (Handle stale pickle cache entries referencing removed classes)
* 22:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|124f4ca}} (Replace wikibase:isSomeValue with STRSTARTS filter for QLever compatibility)
* 21:28 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fd5828f}} (Detect grouping type via SPARQL and flatten to single GroupingConfiguration)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|e9fae82}} (Add AbstractGroupingType base class for grouping type strategies)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fee0ed5}} (Extract GroupingType strategy classes from GroupingConfiguration hierarchy)

=== 2026-03-18 ===
* 20:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f9dfb79}} (Add /healthz endpoint for automatic health check)

=== 2026-03-17 ===
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2940962}} (Replace GROUP_MAPPING Enum with SPECIAL_GROUPINGS tuple)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|420658b}} (Add MARKER attribute to special grouping classes)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|976599f}} (Rename UnknownValueGrouping.TITLE to MARKER)

=== 2026-03-14 ===
* 16:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be09048}} (Add Wikimedia Commons support using QLever endpoint) for [[phab:T294893|T294893]]
* 16:38 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|ae53e61}} (Derive QLever UI URL from engine endpoint)
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5134f77}} (Use ka/Ma unit suffixes for geological-scale year grouping headings) for [[phab:T236590|T236590]]
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8a735a0}} (Add automatic date resolution rebinning for year groupings) for [[phab:T236590|T236590]]
* 15:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|6478051}}, {{Gerrit|4206c09}}, {{Gerrit|0e920fe}}, {{Gerrit|4eefea1}}

=== 2026-03-13 ===
* 20:33 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a39f1d9}} (Add proper handling for transient Wikidata server errors) for [[phab:T415439|T415439]]
* 20:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|0c418a5}} (Fix tests hitting live Wikidata API (and potential rate-limits)) for [[phab:T420054|T420054]]

=== 2026-03-11 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9665d8f}} (Remove obsolete GroupingType enum from column.py)
* 21:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|60d500a}} (Remove backward-compatible formatting wrapper methods from PropertyStatistics)
* 21:20 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a452c28}} (Extract formatting logic into separate ResultsFormatter class)
* 17:29 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|02338eb}} (Move heading strings to class constants in Grouping objects) and {{Gerrit|d730307}} (Format NoGroup and Totals in  with the other groupings)

=== 2026-03-09 ===
* 18:32 wmbot~jeanfred@tools-bastion-15: Recreated Python 3.11 virtual environment using 'toolforge webservice python3.11 shell' / 'webservice-python-bootstrap --fresh' / '/data/project/integraality/www/python/venv/bin/pip install -r /data/project/integraality/integraality/requirements.txt'
* 18:30 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|55f1132}} (Switch to Python 3.11 as default interpreter)

=== 2026-03-08 ===
* 11:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|37fcc25}} (Extract jobs-framework runtime image as its own variable) and {{Gerrit|1f9bfbb}} (Add service.template deployment to Ansible)

=== 2026-03-06 ===
* 10:08 wmbot~jeanfred@tools-bastion-15: Successfully installed flask-3.1.3 mwparserfromhell-0.7.2 pywikibot-10.7.6 redis-7.0.1 (using a webservice python3.9 shell)
* 10:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fa536f}}, {{Gerrit|9e5111c}}, {{Gerrit|e176f18}}, {{Gerrit|eff3045}}, {{Gerrit|8532e32}} (Dependencies upgrades)

=== 2026-03-05 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5075934}} (Add support for user-specified groupings via template parameter) for [[phab:T419059|T419059]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|3c9fa98}} (Remove two debug print statements)
* 18:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28d800e}} (Extract rdfs:label SPARQL fragment from grouping.py to get_label_for_variable utility), {{Gerrit|40c39de}} (Make intermediate label variables unique to avoid clashes in get_label_for_variable) and {{Gerrit|64e8f94}} (Replace SERVICE wikibase:label with native SPARQL for QLever compatibility) for [[phab:T385749|T385749]]

=== 2025-11-03 ===
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]

=== 2025-10-29 ===
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f40974f}} (Add links to QLever in the Queries) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28c2e1b}} (Add link to QLever on queries error page) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4cc2bcc}} (Add configurable SPARQL endpoint support for QLever in PagesProcessor) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be5c234}} (Add support for QLever as alternative SPARQL query engine) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f34624d}} (Move pywikibot exception handling to SPARQL engine class) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4da9f33}} (Implement SPARQL query engine dependency injection) for [[phab:T385749|T385749]]

=== 2025-10-28 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|92ab962}} (Update pywikibot from 10.0.0 to latest 10.6.0)
* 21:34 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|77deb17}} (Remove link to 'SGE-jobs' tool in footer)

=== 2025-10-27 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8db01c6}} (Simplify Flask webserver startup in app.py)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|31f14b0}} (Bump ruff pre-commit hook from 0.9.3 to latest 0.14.2)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|995121d}} (Bump uv pre-commit hook from 0.5.24 to latest 0.9.5 and regenerate requirements files)
* 09:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|69669c0}} (Optimize 'get_grouping_information_query' query using subquery) for [[phab:T400480|T400480]]

=== 2025-10-25 ===
* 09:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9ac4c1f}} (Convert SPARQL query tests to use triple-quoted strings in PropertyStatisticsTest)

=== 2025-10-24 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5511fd0}} (Fix test_get_grouping_information_query_with_grouping_link for YearGrouping)
* 19:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8bb5d37}} (Convert SPARQL query tests to use triple-quoted strings)
* 19:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|792b38e}} (Remove redundant GROUP BY parameter in 'get_grouping_information_query')

=== 2025-05-06 ===
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|284a70a}} (Update pywikibot from 9.6.1 to latest 10.0.0)
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|1f030a9}} (Re-generate uv lockfile to add 'upload-time' markers)
* 11:19 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8f2391e}} (Replace deprecated  with  in two unit tests)

=== 2025-05-04 ===
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7a2fa93}} (Fallback to mul when formatting grouping links) for [[phab:T393166|T393166]]
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)

=== 2025-05-03 ===
* 19:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|781934d}} (Move grouping-link to Grouping objects and retrieve via SPARQL) for [[phab:T237276|T237276]]

=== 2025-05-02 ===
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5bc6fe9}} (Remove unused local variable in GroupingTest.test unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|f5f0044}} (Remove unused local variable in  unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a742a2}} (Remove unused column_key local variable from get_query_for_items_for_property_[posi{{!}}nega]tive)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|620ac00}} (Remove unused patcher in QueriesTests setUp method)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|e32b9b9}} (Allow to optionally pass wikiproject data to SitelinkColumn constructor)

=== 2025-05-01 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|2a91226}} (Catch WDQS Timeouts and raise proper exception for all SPARQL queries) for [[phab:T278156|T278156]]
* 20:29 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|397c5e8}} (Also catch ServerError to handle WDQS timeouts) for [[phab:T384882|T384882]]

=== 2025-04-28 ===
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|eba498e}} (Use correct namespace when using pywikibot exceptions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ea55660}} (Add unit-tests for page_saving.py functions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|58918e6}} (Tweak linebreak in get_grouping_information_query which was breaking unit tests)

=== 2025-01-23 ===
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|a76ceec}} (Switch from Pipenv to uv as project manager)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5941f3f}} (Upgrade all Python dev-dependencies)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix 'QueriesTests' unit tests in test_app)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix  unit tests in test_app)

=== 2025-01-17 ===
* 10:03 wmbot~jeanfred@tools-sgebastion-10: Recreated virtual environment using 'toolforge webservice python3.9 shell' / 'webservice-python-bootstrap --fresh'

=== 2025-01-16 ===
* 16:36 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|525c9ed}} (Upgrade all Python dependencies)

=== 2024-07-01 ===
* 19:54 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8eecfc6}} &amp; {{Gerrit|0466207}}

=== 2024-06-06 ===
* 12:47 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|022fa2f}} ([[phab:T312727|T312727]])
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ec28a13}}
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7adc07f}}

=== 2024-05-10 ===
* 15:57 wmbot~multichill@tools-bastion-12: Leaving project

=== 2024-05-04 ===
* 20:22 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|51ba697}}
* 20:07 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8ec2fec}} ([[phab:T251008|T251008]])

=== 2024-05-03 ===
* 12:01 wmbot~multichill@tools-bastion-12: emails: all -

=== 2024-04-30 ===
* 19:10 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5a91a78}}, {{Gerrit|864d518}}, c9b199 ([[phab:T251008|T251008]]), {{Gerrit|0dd731f}}

=== 2024-03-17 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a93a59}} ([[phab:T319813|T319813]])
* 20:25 wmbot~jeanfred@tools-sgebastion-10: Re-created virtual environment for Python 3.9 ([[phab:T319813|T319813]])
* 19:26 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|bde5715}}, {{Gerrit|9bdfb02}} ([[phab:T319813|T319813]])

=== 2023-11-21 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|992670c}} ([[phab:T351574|T351574]])
* 19:55 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|1f41a79}}, {{Gerrit|38e7623}}

=== 2023-10-17 ===
* 14:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d7f09b3}}, {{Gerrit|eb1de62}}, {{Gerrit|f4e3656}}, {{Gerrit|82f90c4}} ([[phab:T312726|T312726]])

=== 2023-10-03 ===
* 21:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 19:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 18:04 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|da99820}}, {{Gerrit|801ddeb}}, {{Gerrit|f2ffd22}} ([[phab:T312729|T312729]])

=== 2023-10-02 ===
* 09:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3f9a3d5}}

=== 2023-09-26 ===
* 18:25 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3caa9ed}}, {{Gerrit|a8029ed}}, {{Gerrit|c88ad35}}, {{Gerrit|e575508}}, {{Gerrit|3e3f47f}}, {{Gerrit|63ab297}}, {{Gerrit|06cb52d}}, {{Gerrit|ac64840}}, {{Gerrit|6178698}} ([[phab:T312728|T312728]])

=== 2023-06-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5aecae8}}, {{Gerrit|649259f}}, {{Gerrit|9712666}} ([[phab:T338684|T338684]])

=== 2022-12-26 ===
* 10:48 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c90f9ef}} ([[phab:T325936|T325936]])

=== 2022-09-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|bcbd306}}

=== 2022-07-12 ===
* 15:04 wm-bot: &lt;jeanfred&gt; Triggering full update
* 15:02 wm-bot: &lt;jeanfred&gt; jstop {{Gerrit|5976623}} to kill weekly update job stuck since May 22

=== 2022-06-03 ===
* 12:37 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9e85ede}} ([[phab:T309861|T309861]])

=== 2022-05-15 ===
* 21:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3a675b6}}
* 20:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|80f073e}}
* 19:31 wm-bot: &lt;jeanfred&gt; Uninstall removed requirements and restart webservice
* 19:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|f6dc34b}}
* 19:15 wm-bot: &lt;jeanfred&gt; Install all requirements and restart webservice
* 19:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d395b4c}}, {{Gerrit|591c450}}, {{Gerrit|6d09e5b}}

=== 2022-05-14 ===
* 14:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2a521d1}}, {{Gerrit|29d6b62}}, {{Gerrit|8bd793d}} ([[phab:T257942|T257942]])

=== 2022-05-11 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|43f07d2}}, {{Gerrit|da1f2ef}}, {{Gerrit|6ab7547}}, {{Gerrit|5f6b20a}}, {{Gerrit|0df2121}}

=== 2022-01-28 ===
* 20:44 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|13ed7a3}}

=== 2021-11-01 ===
* 20:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|972a1df}}

=== 2021-10-30 ===
* 19:36 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c1628b9}} and {{Gerrit|b6940b0}}

=== 2021-10-29 ===
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|40c8cd3}}
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|af49bab}} ([[phab:T236590|T236590]] and [[phab:T273226|T273226]])
* 09:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2260c8f}} (follow up to [[phab:T294570|T294570]])
* 09:16 wm-bot: &lt;jeanfred&gt; Install requirements in py37 venv ([[phab:T294570|T294570]])
* 09:14 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a4560a}} and {{Gerrit|de296cd}} ([[phab:T294570|T294570]])

=== 2021-10-28 ===
* 17:14 wm-bot: &lt;jeanfred&gt; Rollbacked to {{Gerrit|977236b42daef66818e561b49e64264a0c785454}} for [[phab:T294570|T294570]] and restart
* 12:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a8e7fb}} ([[phab:T236590|T236590]])
* 11:31 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|30a2f2ba}} ([[phab:T278156|T278156]])
* 11:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5c547d8}}
* 10:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b4f12bc}} to help investigate [[phab:T236960|T236960]]
* 09:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|137f5b8}}, {{Gerrit|c816e41}} {{Gerrit|8e0b9ca}}, {{Gerrit|8dbc099}} ([[phab:T273226|T273226]])

=== 2021-10-27 ===
* 09:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6194d84}} ([[phab:T273226|T273226]])

=== 2021-10-25 ===
* 21:41 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|7e3343a}} ([[phab:T278156|T278156]])
* 20:22 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|ef0f537}} ([[phab:T284684|T284684]])
* 20:15 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|977236b}} ([[phab:T284183|T284183]])
* 18:47 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5dfef39}}, {{Gerrit|e940ae7}}, {{Gerrit|765b314}}, {{Gerrit|3e4c2de}}, {{Gerrit|c859c45}}, {{Gerrit|5526cb3}}

=== 2021-06-11 ===
* 15:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|a753cb2}}
* 15:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5187a4d}}

=== 2021-05-31 ===
* 09:20 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|79e458d}}

=== 2021-05-22 ===
* 20:58 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|258aae9a}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|67c42f43}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d170ea1}}
* 20:24 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2f39621}}
* 20:21 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b52079f}}
* 17:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|8ffbad82c3}} ([[phab:T279236|T279236]])
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6de2308}} and {{Gerrit|c436760}} ([[phab:T279236|T279236]])
* 16:06 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0966be04}} ([[phab:T279236|T279236]])
* 15:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9727a01c}}
* 14:02 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|69048ef}}
* 13:00 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|633eeb0f}}
* 12:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d4fa363}} ([[phab:T283287|T283287]])

=== 2021-04-03 ===
* 21:34 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|666222b}} and {{Gerrit|0dcafff}} ([[phab:T279236|T279236]])
* 21:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])
* 16:45 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])

=== 2021-03-26 ===
* 20:57 wm-bot: &lt;jeanfred&gt; Reinstalled requirements in both virtual environments for [[phab:T240312|T240312]]
* 20:55 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|bcc4fe41}} ([[phab:T240312|T240312]])

=== 2020-07-06 ===
* 22:55 wm-bot: &lt;root&gt; Migrated .webservicerc to service.template ([[phab:T257229|T257229]])

=== 2020-06-18 ===
* 11:01 wm-bot: &lt;jeanfred&gt; Restarted with --canonical for Toolforge domain migration

=== 2020-05-01 ===
* 22:59 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|f0db935}}, {{Gerrit|f8d9fdf}}, {{Gerrit|f57c060}}, {{Gerrit|6c534b7}} ([[phab:T248788|T248788]])

=== 2020-04-29 ===
* 20:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eabbac8}}, {{Gerrit|b74b352}} ([[phab:T248788|T248788]])

=== 2020-04-24 ===
* 22:22 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6963040}}, {{Gerrit|35a3ea7}}, {{Gerrit|22e27b5}}, {{Gerrit|9dd44ff}}, {{Gerrit|0e45815}}, {{Gerrit|6f36ffd}} ([[phab:T248788|T248788]])
* 19:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|9fea19b}}, {{Gerrit|f38047e}}

=== 2020-02-25 ===
* 23:35 wm-bot: &lt;root&gt; Migrated to 2020 Kubernetes cluster

=== 2020-02-16 ===
* 16:49 JeanFred: &quot;Deploy latest from Git master: {{Gerrit|0fcb522}} ([[phab:T243780|T243780]]), {{Gerrit|4cb6a5a}} ([[phab:T243780|T243780]], [[phab:T239067|T239067]])&quot;

=== 2020-02-11 ===
* 21:23 wm-bot157: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|c350e4f}} ([[phab:T243998|T243998]]), {{Gerrit|fa12c21}}

=== 2020-02-05 ===
* 12:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8977958}} ([[phab:T243780|T243780]])

=== 2020-02-03 ===
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|3b2867f}}, {{Gerrit|10f67fd}}, {{Gerrit|2e6a669}} ([[phab:T244030|T244030]]), {{Gerrit|279c92b}}, {{Gerrit|de6308e}}

=== 2020-01-16 ===
* 21:44 wm-bot: &lt;jeanfred&gt; Perform webservice restart for [[phab:T242967|T242967]]

=== 2019-12-11 ===
* 15:15 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|257618f1}} ([[phab:T240312|T240312]])

=== 2019-12-02 ===
* 06:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|770cc93}} ([[phab:T237182|T237182]])

=== 2019-11-25 ===
* 11:36 wm-bot: &lt;jeanfred&gt; Trigger a manual update of all dashboards ([[phab:T239085|T239085]])
* 11:34 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|684fb77}} ([[phab:T239085|T239085]])

=== 2019-11-21 ===
* 08:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2f94256}} ([[phab:T237187|T237187]])

=== 2019-11-10 ===
* 15:31 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|ab714de}} ([[phab:T224226|T224226]])

=== 2019-11-06 ===
* 09:12 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|7202182}} ([[phab:T237187|T237187]])
* 09:01 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|29b19c3}} ([[phab:T237182|T237182]])

=== 2019-11-04 ===
* 17:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d0f4222}} ([[phab:T237189|T237189]])
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|588dae8}}
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|471deddd}} ([[phab:T237182|T237182]])

=== 2019-10-31 ===
* 09:06 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|46484f5}} ([[phab:T224226|T224226]])
* 09:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2d71154}}, {{Gerrit|d0d2937}}, {{Gerrit|905007b}}, {{Gerrit|effee36}}, {{Gerrit|7c15083}}

=== 2019-10-30 ===
* 19:42 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|289a41b}}, {{Gerrit|02a1a4f}}, {{Gerrit|e8d4363}} ([[phab:T224226|T224226]])

=== 2019-10-25 ===
* 15:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|82de2957}} ([[phab:T224212|T224212]], [[phab:T223930|T223930]])
* 14:08 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6a055c2}} ([[phab:T228405|T228405]])
* 13:24 wm-bot: &lt;jeanfred&gt; Deploying from master, will break things.

=== 2019-09-19 ===
* 18:38 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d214f09}}, {{Gerrit|9b8dfa5}}
* 17:27 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|12ebc39}}

=== 2019-07-07 ===
* 22:49 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|5b4a1c6}}, {{Gerrit|ab90227}}, {{Gerrit|d101ca1}}, {{Gerrit|394be6a}}, {{Gerrit|ae16d68}}, {{Gerrit|728a8d2}}

=== 2019-05-30 ===
* 12:08 wm-bot: &lt;jeanfred&gt; Nuked the virtualenv and reinstalled all deps from scratch, in desperation for [[phab:T224651|T224651]]
* 10:59 wm-bot: &lt;jeanfred&gt; Service stop, mv logs, service start for [[phab:T224651|T224651]]

=== 2019-05-22 ===
* 16:39 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8e76a8a}}
* 13:13 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2fdede2}}, {{Gerrit|6b6c07c}}, {{Gerrit|c17cb78}} ([[phab:T224090|T224090]])

=== 2019-05-20 ===
* 19:01 wm-bot: &lt;jeanfred&gt; Restart uwsgi with 10 workers rather than the default of 4
* 19:00 wm-bot: &lt;jeanfred&gt; --help


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>i16hibljdvuu6jqtyegeki3hjn7f50a</sha1>
    </revision>
    <revision>
      <id>2408831</id>
      <parentid>2408830</parentid>
      <timestamp>2026-05-03T19:44:22Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>wmbot~jeanfred@tools-bastion-15: Deploy 74bf18f (Add Listeria wikitext formatting to Grouping rows) for T368001</comment>
      <origin>2408831</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="23808" sha1="699eab3ejrr8lk1otttke3krely9b9t" xml:space="preserve">=== 2026-05-03 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|74bf18f}} (Add Listeria wikitext formatting to Grouping rows) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9b32a82}} (Split retrieve_and_process_data in process_page) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4dcc1a0}} (Extract prepare_report_groupings from process_data in PropertyStatistics) for [[phab:T368001|T368001]]
* 19:41 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|928f398}} (Remove obsolete version attribute from docker-compose.yml)

=== 2026-05-01 ===
* 18:16 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|01e97d6}} (Extract grouping link strategies into GroupingLink hierarchy), {{Gerrit|359770d}} (Remove pass-through methods for grouping link in GroupingConfiguration), {{Gerrit|54e4a7b}} (Add lang parameter to get_label_for_variable), {{Gerrit|cccd661}} (Simplify SPARQL for mul language in get_label_for_variable), {{Gerrit|548383d}} (Support any label language for grouping links), {{Gerrit|d571596}} (Support QID-based grouping links), {{Gerrit|cfd7c68}} (
* 18:15 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|19b856d}}, {{Gerrit|473fbeb}}, {{Gerrit|0cb0915}}

=== 2026-03-27 ===
* 22:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2bad8b3}} (Catch ModuleNotFoundError when unpickling stale cache entries)

=== 2026-03-24 ===
* 16:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|7883b73}} (Switch from bare imports to relative imports)
* 15:56 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5dbe412}} (Resolve grouping type eagerly in PropertyStatistics constructor)

=== 2026-03-23 ===
* 22:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|1b60d39}} (Handle stale pickle cache entries referencing removed classes)
* 22:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|124f4ca}} (Replace wikibase:isSomeValue with STRSTARTS filter for QLever compatibility)
* 21:28 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fd5828f}} (Detect grouping type via SPARQL and flatten to single GroupingConfiguration)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|e9fae82}} (Add AbstractGroupingType base class for grouping type strategies)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fee0ed5}} (Extract GroupingType strategy classes from GroupingConfiguration hierarchy)

=== 2026-03-18 ===
* 20:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f9dfb79}} (Add /healthz endpoint for automatic health check)

=== 2026-03-17 ===
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2940962}} (Replace GROUP_MAPPING Enum with SPECIAL_GROUPINGS tuple)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|420658b}} (Add MARKER attribute to special grouping classes)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|976599f}} (Rename UnknownValueGrouping.TITLE to MARKER)

=== 2026-03-14 ===
* 16:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be09048}} (Add Wikimedia Commons support using QLever endpoint) for [[phab:T294893|T294893]]
* 16:38 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|ae53e61}} (Derive QLever UI URL from engine endpoint)
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5134f77}} (Use ka/Ma unit suffixes for geological-scale year grouping headings) for [[phab:T236590|T236590]]
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8a735a0}} (Add automatic date resolution rebinning for year groupings) for [[phab:T236590|T236590]]
* 15:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|6478051}}, {{Gerrit|4206c09}}, {{Gerrit|0e920fe}}, {{Gerrit|4eefea1}}

=== 2026-03-13 ===
* 20:33 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a39f1d9}} (Add proper handling for transient Wikidata server errors) for [[phab:T415439|T415439]]
* 20:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|0c418a5}} (Fix tests hitting live Wikidata API (and potential rate-limits)) for [[phab:T420054|T420054]]

=== 2026-03-11 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9665d8f}} (Remove obsolete GroupingType enum from column.py)
* 21:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|60d500a}} (Remove backward-compatible formatting wrapper methods from PropertyStatistics)
* 21:20 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a452c28}} (Extract formatting logic into separate ResultsFormatter class)
* 17:29 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|02338eb}} (Move heading strings to class constants in Grouping objects) and {{Gerrit|d730307}} (Format NoGroup and Totals in  with the other groupings)

=== 2026-03-09 ===
* 18:32 wmbot~jeanfred@tools-bastion-15: Recreated Python 3.11 virtual environment using 'toolforge webservice python3.11 shell' / 'webservice-python-bootstrap --fresh' / '/data/project/integraality/www/python/venv/bin/pip install -r /data/project/integraality/integraality/requirements.txt'
* 18:30 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|55f1132}} (Switch to Python 3.11 as default interpreter)

=== 2026-03-08 ===
* 11:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|37fcc25}} (Extract jobs-framework runtime image as its own variable) and {{Gerrit|1f9bfbb}} (Add service.template deployment to Ansible)

=== 2026-03-06 ===
* 10:08 wmbot~jeanfred@tools-bastion-15: Successfully installed flask-3.1.3 mwparserfromhell-0.7.2 pywikibot-10.7.6 redis-7.0.1 (using a webservice python3.9 shell)
* 10:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fa536f}}, {{Gerrit|9e5111c}}, {{Gerrit|e176f18}}, {{Gerrit|eff3045}}, {{Gerrit|8532e32}} (Dependencies upgrades)

=== 2026-03-05 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5075934}} (Add support for user-specified groupings via template parameter) for [[phab:T419059|T419059]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|3c9fa98}} (Remove two debug print statements)
* 18:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28d800e}} (Extract rdfs:label SPARQL fragment from grouping.py to get_label_for_variable utility), {{Gerrit|40c39de}} (Make intermediate label variables unique to avoid clashes in get_label_for_variable) and {{Gerrit|64e8f94}} (Replace SERVICE wikibase:label with native SPARQL for QLever compatibility) for [[phab:T385749|T385749]]

=== 2025-11-03 ===
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]

=== 2025-10-29 ===
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f40974f}} (Add links to QLever in the Queries) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28c2e1b}} (Add link to QLever on queries error page) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4cc2bcc}} (Add configurable SPARQL endpoint support for QLever in PagesProcessor) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be5c234}} (Add support for QLever as alternative SPARQL query engine) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f34624d}} (Move pywikibot exception handling to SPARQL engine class) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4da9f33}} (Implement SPARQL query engine dependency injection) for [[phab:T385749|T385749]]

=== 2025-10-28 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|92ab962}} (Update pywikibot from 10.0.0 to latest 10.6.0)
* 21:34 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|77deb17}} (Remove link to 'SGE-jobs' tool in footer)

=== 2025-10-27 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8db01c6}} (Simplify Flask webserver startup in app.py)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|31f14b0}} (Bump ruff pre-commit hook from 0.9.3 to latest 0.14.2)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|995121d}} (Bump uv pre-commit hook from 0.5.24 to latest 0.9.5 and regenerate requirements files)
* 09:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|69669c0}} (Optimize 'get_grouping_information_query' query using subquery) for [[phab:T400480|T400480]]

=== 2025-10-25 ===
* 09:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9ac4c1f}} (Convert SPARQL query tests to use triple-quoted strings in PropertyStatisticsTest)

=== 2025-10-24 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5511fd0}} (Fix test_get_grouping_information_query_with_grouping_link for YearGrouping)
* 19:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8bb5d37}} (Convert SPARQL query tests to use triple-quoted strings)
* 19:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|792b38e}} (Remove redundant GROUP BY parameter in 'get_grouping_information_query')

=== 2025-05-06 ===
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|284a70a}} (Update pywikibot from 9.6.1 to latest 10.0.0)
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|1f030a9}} (Re-generate uv lockfile to add 'upload-time' markers)
* 11:19 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8f2391e}} (Replace deprecated  with  in two unit tests)

=== 2025-05-04 ===
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7a2fa93}} (Fallback to mul when formatting grouping links) for [[phab:T393166|T393166]]
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)

=== 2025-05-03 ===
* 19:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|781934d}} (Move grouping-link to Grouping objects and retrieve via SPARQL) for [[phab:T237276|T237276]]

=== 2025-05-02 ===
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5bc6fe9}} (Remove unused local variable in GroupingTest.test unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|f5f0044}} (Remove unused local variable in  unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a742a2}} (Remove unused column_key local variable from get_query_for_items_for_property_[posi{{!}}nega]tive)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|620ac00}} (Remove unused patcher in QueriesTests setUp method)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|e32b9b9}} (Allow to optionally pass wikiproject data to SitelinkColumn constructor)

=== 2025-05-01 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|2a91226}} (Catch WDQS Timeouts and raise proper exception for all SPARQL queries) for [[phab:T278156|T278156]]
* 20:29 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|397c5e8}} (Also catch ServerError to handle WDQS timeouts) for [[phab:T384882|T384882]]

=== 2025-04-28 ===
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|eba498e}} (Use correct namespace when using pywikibot exceptions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ea55660}} (Add unit-tests for page_saving.py functions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|58918e6}} (Tweak linebreak in get_grouping_information_query which was breaking unit tests)

=== 2025-01-23 ===
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|a76ceec}} (Switch from Pipenv to uv as project manager)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5941f3f}} (Upgrade all Python dev-dependencies)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix 'QueriesTests' unit tests in test_app)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix  unit tests in test_app)

=== 2025-01-17 ===
* 10:03 wmbot~jeanfred@tools-sgebastion-10: Recreated virtual environment using 'toolforge webservice python3.9 shell' / 'webservice-python-bootstrap --fresh'

=== 2025-01-16 ===
* 16:36 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|525c9ed}} (Upgrade all Python dependencies)

=== 2024-07-01 ===
* 19:54 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8eecfc6}} &amp; {{Gerrit|0466207}}

=== 2024-06-06 ===
* 12:47 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|022fa2f}} ([[phab:T312727|T312727]])
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ec28a13}}
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7adc07f}}

=== 2024-05-10 ===
* 15:57 wmbot~multichill@tools-bastion-12: Leaving project

=== 2024-05-04 ===
* 20:22 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|51ba697}}
* 20:07 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8ec2fec}} ([[phab:T251008|T251008]])

=== 2024-05-03 ===
* 12:01 wmbot~multichill@tools-bastion-12: emails: all -

=== 2024-04-30 ===
* 19:10 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5a91a78}}, {{Gerrit|864d518}}, c9b199 ([[phab:T251008|T251008]]), {{Gerrit|0dd731f}}

=== 2024-03-17 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a93a59}} ([[phab:T319813|T319813]])
* 20:25 wmbot~jeanfred@tools-sgebastion-10: Re-created virtual environment for Python 3.9 ([[phab:T319813|T319813]])
* 19:26 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|bde5715}}, {{Gerrit|9bdfb02}} ([[phab:T319813|T319813]])

=== 2023-11-21 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|992670c}} ([[phab:T351574|T351574]])
* 19:55 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|1f41a79}}, {{Gerrit|38e7623}}

=== 2023-10-17 ===
* 14:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d7f09b3}}, {{Gerrit|eb1de62}}, {{Gerrit|f4e3656}}, {{Gerrit|82f90c4}} ([[phab:T312726|T312726]])

=== 2023-10-03 ===
* 21:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 19:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 18:04 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|da99820}}, {{Gerrit|801ddeb}}, {{Gerrit|f2ffd22}} ([[phab:T312729|T312729]])

=== 2023-10-02 ===
* 09:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3f9a3d5}}

=== 2023-09-26 ===
* 18:25 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3caa9ed}}, {{Gerrit|a8029ed}}, {{Gerrit|c88ad35}}, {{Gerrit|e575508}}, {{Gerrit|3e3f47f}}, {{Gerrit|63ab297}}, {{Gerrit|06cb52d}}, {{Gerrit|ac64840}}, {{Gerrit|6178698}} ([[phab:T312728|T312728]])

=== 2023-06-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5aecae8}}, {{Gerrit|649259f}}, {{Gerrit|9712666}} ([[phab:T338684|T338684]])

=== 2022-12-26 ===
* 10:48 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c90f9ef}} ([[phab:T325936|T325936]])

=== 2022-09-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|bcbd306}}

=== 2022-07-12 ===
* 15:04 wm-bot: &lt;jeanfred&gt; Triggering full update
* 15:02 wm-bot: &lt;jeanfred&gt; jstop {{Gerrit|5976623}} to kill weekly update job stuck since May 22

=== 2022-06-03 ===
* 12:37 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9e85ede}} ([[phab:T309861|T309861]])

=== 2022-05-15 ===
* 21:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3a675b6}}
* 20:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|80f073e}}
* 19:31 wm-bot: &lt;jeanfred&gt; Uninstall removed requirements and restart webservice
* 19:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|f6dc34b}}
* 19:15 wm-bot: &lt;jeanfred&gt; Install all requirements and restart webservice
* 19:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d395b4c}}, {{Gerrit|591c450}}, {{Gerrit|6d09e5b}}

=== 2022-05-14 ===
* 14:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2a521d1}}, {{Gerrit|29d6b62}}, {{Gerrit|8bd793d}} ([[phab:T257942|T257942]])

=== 2022-05-11 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|43f07d2}}, {{Gerrit|da1f2ef}}, {{Gerrit|6ab7547}}, {{Gerrit|5f6b20a}}, {{Gerrit|0df2121}}

=== 2022-01-28 ===
* 20:44 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|13ed7a3}}

=== 2021-11-01 ===
* 20:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|972a1df}}

=== 2021-10-30 ===
* 19:36 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c1628b9}} and {{Gerrit|b6940b0}}

=== 2021-10-29 ===
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|40c8cd3}}
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|af49bab}} ([[phab:T236590|T236590]] and [[phab:T273226|T273226]])
* 09:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2260c8f}} (follow up to [[phab:T294570|T294570]])
* 09:16 wm-bot: &lt;jeanfred&gt; Install requirements in py37 venv ([[phab:T294570|T294570]])
* 09:14 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a4560a}} and {{Gerrit|de296cd}} ([[phab:T294570|T294570]])

=== 2021-10-28 ===
* 17:14 wm-bot: &lt;jeanfred&gt; Rollbacked to {{Gerrit|977236b42daef66818e561b49e64264a0c785454}} for [[phab:T294570|T294570]] and restart
* 12:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a8e7fb}} ([[phab:T236590|T236590]])
* 11:31 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|30a2f2ba}} ([[phab:T278156|T278156]])
* 11:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5c547d8}}
* 10:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b4f12bc}} to help investigate [[phab:T236960|T236960]]
* 09:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|137f5b8}}, {{Gerrit|c816e41}} {{Gerrit|8e0b9ca}}, {{Gerrit|8dbc099}} ([[phab:T273226|T273226]])

=== 2021-10-27 ===
* 09:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6194d84}} ([[phab:T273226|T273226]])

=== 2021-10-25 ===
* 21:41 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|7e3343a}} ([[phab:T278156|T278156]])
* 20:22 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|ef0f537}} ([[phab:T284684|T284684]])
* 20:15 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|977236b}} ([[phab:T284183|T284183]])
* 18:47 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5dfef39}}, {{Gerrit|e940ae7}}, {{Gerrit|765b314}}, {{Gerrit|3e4c2de}}, {{Gerrit|c859c45}}, {{Gerrit|5526cb3}}

=== 2021-06-11 ===
* 15:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|a753cb2}}
* 15:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5187a4d}}

=== 2021-05-31 ===
* 09:20 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|79e458d}}

=== 2021-05-22 ===
* 20:58 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|258aae9a}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|67c42f43}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d170ea1}}
* 20:24 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2f39621}}
* 20:21 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b52079f}}
* 17:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|8ffbad82c3}} ([[phab:T279236|T279236]])
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6de2308}} and {{Gerrit|c436760}} ([[phab:T279236|T279236]])
* 16:06 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0966be04}} ([[phab:T279236|T279236]])
* 15:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9727a01c}}
* 14:02 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|69048ef}}
* 13:00 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|633eeb0f}}
* 12:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d4fa363}} ([[phab:T283287|T283287]])

=== 2021-04-03 ===
* 21:34 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|666222b}} and {{Gerrit|0dcafff}} ([[phab:T279236|T279236]])
* 21:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])
* 16:45 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])

=== 2021-03-26 ===
* 20:57 wm-bot: &lt;jeanfred&gt; Reinstalled requirements in both virtual environments for [[phab:T240312|T240312]]
* 20:55 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|bcc4fe41}} ([[phab:T240312|T240312]])

=== 2020-07-06 ===
* 22:55 wm-bot: &lt;root&gt; Migrated .webservicerc to service.template ([[phab:T257229|T257229]])

=== 2020-06-18 ===
* 11:01 wm-bot: &lt;jeanfred&gt; Restarted with --canonical for Toolforge domain migration

=== 2020-05-01 ===
* 22:59 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|f0db935}}, {{Gerrit|f8d9fdf}}, {{Gerrit|f57c060}}, {{Gerrit|6c534b7}} ([[phab:T248788|T248788]])

=== 2020-04-29 ===
* 20:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eabbac8}}, {{Gerrit|b74b352}} ([[phab:T248788|T248788]])

=== 2020-04-24 ===
* 22:22 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6963040}}, {{Gerrit|35a3ea7}}, {{Gerrit|22e27b5}}, {{Gerrit|9dd44ff}}, {{Gerrit|0e45815}}, {{Gerrit|6f36ffd}} ([[phab:T248788|T248788]])
* 19:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|9fea19b}}, {{Gerrit|f38047e}}

=== 2020-02-25 ===
* 23:35 wm-bot: &lt;root&gt; Migrated to 2020 Kubernetes cluster

=== 2020-02-16 ===
* 16:49 JeanFred: &quot;Deploy latest from Git master: {{Gerrit|0fcb522}} ([[phab:T243780|T243780]]), {{Gerrit|4cb6a5a}} ([[phab:T243780|T243780]], [[phab:T239067|T239067]])&quot;

=== 2020-02-11 ===
* 21:23 wm-bot157: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|c350e4f}} ([[phab:T243998|T243998]]), {{Gerrit|fa12c21}}

=== 2020-02-05 ===
* 12:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8977958}} ([[phab:T243780|T243780]])

=== 2020-02-03 ===
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|3b2867f}}, {{Gerrit|10f67fd}}, {{Gerrit|2e6a669}} ([[phab:T244030|T244030]]), {{Gerrit|279c92b}}, {{Gerrit|de6308e}}

=== 2020-01-16 ===
* 21:44 wm-bot: &lt;jeanfred&gt; Perform webservice restart for [[phab:T242967|T242967]]

=== 2019-12-11 ===
* 15:15 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|257618f1}} ([[phab:T240312|T240312]])

=== 2019-12-02 ===
* 06:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|770cc93}} ([[phab:T237182|T237182]])

=== 2019-11-25 ===
* 11:36 wm-bot: &lt;jeanfred&gt; Trigger a manual update of all dashboards ([[phab:T239085|T239085]])
* 11:34 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|684fb77}} ([[phab:T239085|T239085]])

=== 2019-11-21 ===
* 08:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2f94256}} ([[phab:T237187|T237187]])

=== 2019-11-10 ===
* 15:31 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|ab714de}} ([[phab:T224226|T224226]])

=== 2019-11-06 ===
* 09:12 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|7202182}} ([[phab:T237187|T237187]])
* 09:01 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|29b19c3}} ([[phab:T237182|T237182]])

=== 2019-11-04 ===
* 17:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d0f4222}} ([[phab:T237189|T237189]])
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|588dae8}}
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|471deddd}} ([[phab:T237182|T237182]])

=== 2019-10-31 ===
* 09:06 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|46484f5}} ([[phab:T224226|T224226]])
* 09:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2d71154}}, {{Gerrit|d0d2937}}, {{Gerrit|905007b}}, {{Gerrit|effee36}}, {{Gerrit|7c15083}}

=== 2019-10-30 ===
* 19:42 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|289a41b}}, {{Gerrit|02a1a4f}}, {{Gerrit|e8d4363}} ([[phab:T224226|T224226]])

=== 2019-10-25 ===
* 15:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|82de2957}} ([[phab:T224212|T224212]], [[phab:T223930|T223930]])
* 14:08 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6a055c2}} ([[phab:T228405|T228405]])
* 13:24 wm-bot: &lt;jeanfred&gt; Deploying from master, will break things.

=== 2019-09-19 ===
* 18:38 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d214f09}}, {{Gerrit|9b8dfa5}}
* 17:27 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|12ebc39}}

=== 2019-07-07 ===
* 22:49 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|5b4a1c6}}, {{Gerrit|ab90227}}, {{Gerrit|d101ca1}}, {{Gerrit|394be6a}}, {{Gerrit|ae16d68}}, {{Gerrit|728a8d2}}

=== 2019-05-30 ===
* 12:08 wm-bot: &lt;jeanfred&gt; Nuked the virtualenv and reinstalled all deps from scratch, in desperation for [[phab:T224651|T224651]]
* 10:59 wm-bot: &lt;jeanfred&gt; Service stop, mv logs, service start for [[phab:T224651|T224651]]

=== 2019-05-22 ===
* 16:39 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8e76a8a}}
* 13:13 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2fdede2}}, {{Gerrit|6b6c07c}}, {{Gerrit|c17cb78}} ([[phab:T224090|T224090]])

=== 2019-05-20 ===
* 19:01 wm-bot: &lt;jeanfred&gt; Restart uwsgi with 10 workers rather than the default of 4
* 19:00 wm-bot: &lt;jeanfred&gt; --help


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>699eab3ejrr8lk1otttke3krely9b9t</sha1>
    </revision>
    <revision>
      <id>2408832</id>
      <parentid>2408831</parentid>
      <timestamp>2026-05-03T19:44:24Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>wmbot~jeanfred@tools-bastion-15: Deploy 729decc (Add GroupingPageCreator to create Listeria subpages) for T368001</comment>
      <origin>2408832</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="23958" sha1="ewiyu4u5e4ds6q9g5s7heiriyl98w8p" xml:space="preserve">=== 2026-05-03 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|729decc}} (Add GroupingPageCreator to create Listeria subpages) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|74bf18f}} (Add Listeria wikitext formatting to Grouping rows) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9b32a82}} (Split retrieve_and_process_data in process_page) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4dcc1a0}} (Extract prepare_report_groupings from process_data in PropertyStatistics) for [[phab:T368001|T368001]]
* 19:41 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|928f398}} (Remove obsolete version attribute from docker-compose.yml)

=== 2026-05-01 ===
* 18:16 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|01e97d6}} (Extract grouping link strategies into GroupingLink hierarchy), {{Gerrit|359770d}} (Remove pass-through methods for grouping link in GroupingConfiguration), {{Gerrit|54e4a7b}} (Add lang parameter to get_label_for_variable), {{Gerrit|cccd661}} (Simplify SPARQL for mul language in get_label_for_variable), {{Gerrit|548383d}} (Support any label language for grouping links), {{Gerrit|d571596}} (Support QID-based grouping links), {{Gerrit|cfd7c68}} (
* 18:15 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|19b856d}}, {{Gerrit|473fbeb}}, {{Gerrit|0cb0915}}

=== 2026-03-27 ===
* 22:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2bad8b3}} (Catch ModuleNotFoundError when unpickling stale cache entries)

=== 2026-03-24 ===
* 16:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|7883b73}} (Switch from bare imports to relative imports)
* 15:56 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5dbe412}} (Resolve grouping type eagerly in PropertyStatistics constructor)

=== 2026-03-23 ===
* 22:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|1b60d39}} (Handle stale pickle cache entries referencing removed classes)
* 22:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|124f4ca}} (Replace wikibase:isSomeValue with STRSTARTS filter for QLever compatibility)
* 21:28 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fd5828f}} (Detect grouping type via SPARQL and flatten to single GroupingConfiguration)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|e9fae82}} (Add AbstractGroupingType base class for grouping type strategies)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fee0ed5}} (Extract GroupingType strategy classes from GroupingConfiguration hierarchy)

=== 2026-03-18 ===
* 20:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f9dfb79}} (Add /healthz endpoint for automatic health check)

=== 2026-03-17 ===
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2940962}} (Replace GROUP_MAPPING Enum with SPECIAL_GROUPINGS tuple)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|420658b}} (Add MARKER attribute to special grouping classes)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|976599f}} (Rename UnknownValueGrouping.TITLE to MARKER)

=== 2026-03-14 ===
* 16:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be09048}} (Add Wikimedia Commons support using QLever endpoint) for [[phab:T294893|T294893]]
* 16:38 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|ae53e61}} (Derive QLever UI URL from engine endpoint)
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5134f77}} (Use ka/Ma unit suffixes for geological-scale year grouping headings) for [[phab:T236590|T236590]]
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8a735a0}} (Add automatic date resolution rebinning for year groupings) for [[phab:T236590|T236590]]
* 15:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|6478051}}, {{Gerrit|4206c09}}, {{Gerrit|0e920fe}}, {{Gerrit|4eefea1}}

=== 2026-03-13 ===
* 20:33 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a39f1d9}} (Add proper handling for transient Wikidata server errors) for [[phab:T415439|T415439]]
* 20:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|0c418a5}} (Fix tests hitting live Wikidata API (and potential rate-limits)) for [[phab:T420054|T420054]]

=== 2026-03-11 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9665d8f}} (Remove obsolete GroupingType enum from column.py)
* 21:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|60d500a}} (Remove backward-compatible formatting wrapper methods from PropertyStatistics)
* 21:20 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a452c28}} (Extract formatting logic into separate ResultsFormatter class)
* 17:29 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|02338eb}} (Move heading strings to class constants in Grouping objects) and {{Gerrit|d730307}} (Format NoGroup and Totals in  with the other groupings)

=== 2026-03-09 ===
* 18:32 wmbot~jeanfred@tools-bastion-15: Recreated Python 3.11 virtual environment using 'toolforge webservice python3.11 shell' / 'webservice-python-bootstrap --fresh' / '/data/project/integraality/www/python/venv/bin/pip install -r /data/project/integraality/integraality/requirements.txt'
* 18:30 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|55f1132}} (Switch to Python 3.11 as default interpreter)

=== 2026-03-08 ===
* 11:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|37fcc25}} (Extract jobs-framework runtime image as its own variable) and {{Gerrit|1f9bfbb}} (Add service.template deployment to Ansible)

=== 2026-03-06 ===
* 10:08 wmbot~jeanfred@tools-bastion-15: Successfully installed flask-3.1.3 mwparserfromhell-0.7.2 pywikibot-10.7.6 redis-7.0.1 (using a webservice python3.9 shell)
* 10:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fa536f}}, {{Gerrit|9e5111c}}, {{Gerrit|e176f18}}, {{Gerrit|eff3045}}, {{Gerrit|8532e32}} (Dependencies upgrades)

=== 2026-03-05 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5075934}} (Add support for user-specified groupings via template parameter) for [[phab:T419059|T419059]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|3c9fa98}} (Remove two debug print statements)
* 18:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28d800e}} (Extract rdfs:label SPARQL fragment from grouping.py to get_label_for_variable utility), {{Gerrit|40c39de}} (Make intermediate label variables unique to avoid clashes in get_label_for_variable) and {{Gerrit|64e8f94}} (Replace SERVICE wikibase:label with native SPARQL for QLever compatibility) for [[phab:T385749|T385749]]

=== 2025-11-03 ===
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]

=== 2025-10-29 ===
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f40974f}} (Add links to QLever in the Queries) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28c2e1b}} (Add link to QLever on queries error page) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4cc2bcc}} (Add configurable SPARQL endpoint support for QLever in PagesProcessor) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be5c234}} (Add support for QLever as alternative SPARQL query engine) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f34624d}} (Move pywikibot exception handling to SPARQL engine class) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4da9f33}} (Implement SPARQL query engine dependency injection) for [[phab:T385749|T385749]]

=== 2025-10-28 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|92ab962}} (Update pywikibot from 10.0.0 to latest 10.6.0)
* 21:34 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|77deb17}} (Remove link to 'SGE-jobs' tool in footer)

=== 2025-10-27 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8db01c6}} (Simplify Flask webserver startup in app.py)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|31f14b0}} (Bump ruff pre-commit hook from 0.9.3 to latest 0.14.2)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|995121d}} (Bump uv pre-commit hook from 0.5.24 to latest 0.9.5 and regenerate requirements files)
* 09:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|69669c0}} (Optimize 'get_grouping_information_query' query using subquery) for [[phab:T400480|T400480]]

=== 2025-10-25 ===
* 09:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9ac4c1f}} (Convert SPARQL query tests to use triple-quoted strings in PropertyStatisticsTest)

=== 2025-10-24 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5511fd0}} (Fix test_get_grouping_information_query_with_grouping_link for YearGrouping)
* 19:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8bb5d37}} (Convert SPARQL query tests to use triple-quoted strings)
* 19:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|792b38e}} (Remove redundant GROUP BY parameter in 'get_grouping_information_query')

=== 2025-05-06 ===
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|284a70a}} (Update pywikibot from 9.6.1 to latest 10.0.0)
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|1f030a9}} (Re-generate uv lockfile to add 'upload-time' markers)
* 11:19 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8f2391e}} (Replace deprecated  with  in two unit tests)

=== 2025-05-04 ===
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7a2fa93}} (Fallback to mul when formatting grouping links) for [[phab:T393166|T393166]]
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)

=== 2025-05-03 ===
* 19:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|781934d}} (Move grouping-link to Grouping objects and retrieve via SPARQL) for [[phab:T237276|T237276]]

=== 2025-05-02 ===
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5bc6fe9}} (Remove unused local variable in GroupingTest.test unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|f5f0044}} (Remove unused local variable in  unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a742a2}} (Remove unused column_key local variable from get_query_for_items_for_property_[posi{{!}}nega]tive)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|620ac00}} (Remove unused patcher in QueriesTests setUp method)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|e32b9b9}} (Allow to optionally pass wikiproject data to SitelinkColumn constructor)

=== 2025-05-01 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|2a91226}} (Catch WDQS Timeouts and raise proper exception for all SPARQL queries) for [[phab:T278156|T278156]]
* 20:29 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|397c5e8}} (Also catch ServerError to handle WDQS timeouts) for [[phab:T384882|T384882]]

=== 2025-04-28 ===
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|eba498e}} (Use correct namespace when using pywikibot exceptions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ea55660}} (Add unit-tests for page_saving.py functions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|58918e6}} (Tweak linebreak in get_grouping_information_query which was breaking unit tests)

=== 2025-01-23 ===
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|a76ceec}} (Switch from Pipenv to uv as project manager)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5941f3f}} (Upgrade all Python dev-dependencies)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix 'QueriesTests' unit tests in test_app)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix  unit tests in test_app)

=== 2025-01-17 ===
* 10:03 wmbot~jeanfred@tools-sgebastion-10: Recreated virtual environment using 'toolforge webservice python3.9 shell' / 'webservice-python-bootstrap --fresh'

=== 2025-01-16 ===
* 16:36 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|525c9ed}} (Upgrade all Python dependencies)

=== 2024-07-01 ===
* 19:54 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8eecfc6}} &amp; {{Gerrit|0466207}}

=== 2024-06-06 ===
* 12:47 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|022fa2f}} ([[phab:T312727|T312727]])
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ec28a13}}
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7adc07f}}

=== 2024-05-10 ===
* 15:57 wmbot~multichill@tools-bastion-12: Leaving project

=== 2024-05-04 ===
* 20:22 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|51ba697}}
* 20:07 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8ec2fec}} ([[phab:T251008|T251008]])

=== 2024-05-03 ===
* 12:01 wmbot~multichill@tools-bastion-12: emails: all -

=== 2024-04-30 ===
* 19:10 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5a91a78}}, {{Gerrit|864d518}}, c9b199 ([[phab:T251008|T251008]]), {{Gerrit|0dd731f}}

=== 2024-03-17 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a93a59}} ([[phab:T319813|T319813]])
* 20:25 wmbot~jeanfred@tools-sgebastion-10: Re-created virtual environment for Python 3.9 ([[phab:T319813|T319813]])
* 19:26 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|bde5715}}, {{Gerrit|9bdfb02}} ([[phab:T319813|T319813]])

=== 2023-11-21 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|992670c}} ([[phab:T351574|T351574]])
* 19:55 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|1f41a79}}, {{Gerrit|38e7623}}

=== 2023-10-17 ===
* 14:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d7f09b3}}, {{Gerrit|eb1de62}}, {{Gerrit|f4e3656}}, {{Gerrit|82f90c4}} ([[phab:T312726|T312726]])

=== 2023-10-03 ===
* 21:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 19:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 18:04 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|da99820}}, {{Gerrit|801ddeb}}, {{Gerrit|f2ffd22}} ([[phab:T312729|T312729]])

=== 2023-10-02 ===
* 09:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3f9a3d5}}

=== 2023-09-26 ===
* 18:25 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3caa9ed}}, {{Gerrit|a8029ed}}, {{Gerrit|c88ad35}}, {{Gerrit|e575508}}, {{Gerrit|3e3f47f}}, {{Gerrit|63ab297}}, {{Gerrit|06cb52d}}, {{Gerrit|ac64840}}, {{Gerrit|6178698}} ([[phab:T312728|T312728]])

=== 2023-06-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5aecae8}}, {{Gerrit|649259f}}, {{Gerrit|9712666}} ([[phab:T338684|T338684]])

=== 2022-12-26 ===
* 10:48 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c90f9ef}} ([[phab:T325936|T325936]])

=== 2022-09-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|bcbd306}}

=== 2022-07-12 ===
* 15:04 wm-bot: &lt;jeanfred&gt; Triggering full update
* 15:02 wm-bot: &lt;jeanfred&gt; jstop {{Gerrit|5976623}} to kill weekly update job stuck since May 22

=== 2022-06-03 ===
* 12:37 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9e85ede}} ([[phab:T309861|T309861]])

=== 2022-05-15 ===
* 21:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3a675b6}}
* 20:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|80f073e}}
* 19:31 wm-bot: &lt;jeanfred&gt; Uninstall removed requirements and restart webservice
* 19:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|f6dc34b}}
* 19:15 wm-bot: &lt;jeanfred&gt; Install all requirements and restart webservice
* 19:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d395b4c}}, {{Gerrit|591c450}}, {{Gerrit|6d09e5b}}

=== 2022-05-14 ===
* 14:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2a521d1}}, {{Gerrit|29d6b62}}, {{Gerrit|8bd793d}} ([[phab:T257942|T257942]])

=== 2022-05-11 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|43f07d2}}, {{Gerrit|da1f2ef}}, {{Gerrit|6ab7547}}, {{Gerrit|5f6b20a}}, {{Gerrit|0df2121}}

=== 2022-01-28 ===
* 20:44 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|13ed7a3}}

=== 2021-11-01 ===
* 20:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|972a1df}}

=== 2021-10-30 ===
* 19:36 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c1628b9}} and {{Gerrit|b6940b0}}

=== 2021-10-29 ===
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|40c8cd3}}
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|af49bab}} ([[phab:T236590|T236590]] and [[phab:T273226|T273226]])
* 09:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2260c8f}} (follow up to [[phab:T294570|T294570]])
* 09:16 wm-bot: &lt;jeanfred&gt; Install requirements in py37 venv ([[phab:T294570|T294570]])
* 09:14 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a4560a}} and {{Gerrit|de296cd}} ([[phab:T294570|T294570]])

=== 2021-10-28 ===
* 17:14 wm-bot: &lt;jeanfred&gt; Rollbacked to {{Gerrit|977236b42daef66818e561b49e64264a0c785454}} for [[phab:T294570|T294570]] and restart
* 12:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a8e7fb}} ([[phab:T236590|T236590]])
* 11:31 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|30a2f2ba}} ([[phab:T278156|T278156]])
* 11:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5c547d8}}
* 10:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b4f12bc}} to help investigate [[phab:T236960|T236960]]
* 09:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|137f5b8}}, {{Gerrit|c816e41}} {{Gerrit|8e0b9ca}}, {{Gerrit|8dbc099}} ([[phab:T273226|T273226]])

=== 2021-10-27 ===
* 09:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6194d84}} ([[phab:T273226|T273226]])

=== 2021-10-25 ===
* 21:41 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|7e3343a}} ([[phab:T278156|T278156]])
* 20:22 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|ef0f537}} ([[phab:T284684|T284684]])
* 20:15 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|977236b}} ([[phab:T284183|T284183]])
* 18:47 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5dfef39}}, {{Gerrit|e940ae7}}, {{Gerrit|765b314}}, {{Gerrit|3e4c2de}}, {{Gerrit|c859c45}}, {{Gerrit|5526cb3}}

=== 2021-06-11 ===
* 15:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|a753cb2}}
* 15:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5187a4d}}

=== 2021-05-31 ===
* 09:20 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|79e458d}}

=== 2021-05-22 ===
* 20:58 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|258aae9a}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|67c42f43}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d170ea1}}
* 20:24 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2f39621}}
* 20:21 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b52079f}}
* 17:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|8ffbad82c3}} ([[phab:T279236|T279236]])
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6de2308}} and {{Gerrit|c436760}} ([[phab:T279236|T279236]])
* 16:06 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0966be04}} ([[phab:T279236|T279236]])
* 15:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9727a01c}}
* 14:02 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|69048ef}}
* 13:00 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|633eeb0f}}
* 12:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d4fa363}} ([[phab:T283287|T283287]])

=== 2021-04-03 ===
* 21:34 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|666222b}} and {{Gerrit|0dcafff}} ([[phab:T279236|T279236]])
* 21:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])
* 16:45 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])

=== 2021-03-26 ===
* 20:57 wm-bot: &lt;jeanfred&gt; Reinstalled requirements in both virtual environments for [[phab:T240312|T240312]]
* 20:55 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|bcc4fe41}} ([[phab:T240312|T240312]])

=== 2020-07-06 ===
* 22:55 wm-bot: &lt;root&gt; Migrated .webservicerc to service.template ([[phab:T257229|T257229]])

=== 2020-06-18 ===
* 11:01 wm-bot: &lt;jeanfred&gt; Restarted with --canonical for Toolforge domain migration

=== 2020-05-01 ===
* 22:59 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|f0db935}}, {{Gerrit|f8d9fdf}}, {{Gerrit|f57c060}}, {{Gerrit|6c534b7}} ([[phab:T248788|T248788]])

=== 2020-04-29 ===
* 20:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eabbac8}}, {{Gerrit|b74b352}} ([[phab:T248788|T248788]])

=== 2020-04-24 ===
* 22:22 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6963040}}, {{Gerrit|35a3ea7}}, {{Gerrit|22e27b5}}, {{Gerrit|9dd44ff}}, {{Gerrit|0e45815}}, {{Gerrit|6f36ffd}} ([[phab:T248788|T248788]])
* 19:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|9fea19b}}, {{Gerrit|f38047e}}

=== 2020-02-25 ===
* 23:35 wm-bot: &lt;root&gt; Migrated to 2020 Kubernetes cluster

=== 2020-02-16 ===
* 16:49 JeanFred: &quot;Deploy latest from Git master: {{Gerrit|0fcb522}} ([[phab:T243780|T243780]]), {{Gerrit|4cb6a5a}} ([[phab:T243780|T243780]], [[phab:T239067|T239067]])&quot;

=== 2020-02-11 ===
* 21:23 wm-bot157: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|c350e4f}} ([[phab:T243998|T243998]]), {{Gerrit|fa12c21}}

=== 2020-02-05 ===
* 12:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8977958}} ([[phab:T243780|T243780]])

=== 2020-02-03 ===
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|3b2867f}}, {{Gerrit|10f67fd}}, {{Gerrit|2e6a669}} ([[phab:T244030|T244030]]), {{Gerrit|279c92b}}, {{Gerrit|de6308e}}

=== 2020-01-16 ===
* 21:44 wm-bot: &lt;jeanfred&gt; Perform webservice restart for [[phab:T242967|T242967]]

=== 2019-12-11 ===
* 15:15 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|257618f1}} ([[phab:T240312|T240312]])

=== 2019-12-02 ===
* 06:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|770cc93}} ([[phab:T237182|T237182]])

=== 2019-11-25 ===
* 11:36 wm-bot: &lt;jeanfred&gt; Trigger a manual update of all dashboards ([[phab:T239085|T239085]])
* 11:34 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|684fb77}} ([[phab:T239085|T239085]])

=== 2019-11-21 ===
* 08:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2f94256}} ([[phab:T237187|T237187]])

=== 2019-11-10 ===
* 15:31 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|ab714de}} ([[phab:T224226|T224226]])

=== 2019-11-06 ===
* 09:12 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|7202182}} ([[phab:T237187|T237187]])
* 09:01 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|29b19c3}} ([[phab:T237182|T237182]])

=== 2019-11-04 ===
* 17:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d0f4222}} ([[phab:T237189|T237189]])
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|588dae8}}
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|471deddd}} ([[phab:T237182|T237182]])

=== 2019-10-31 ===
* 09:06 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|46484f5}} ([[phab:T224226|T224226]])
* 09:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2d71154}}, {{Gerrit|d0d2937}}, {{Gerrit|905007b}}, {{Gerrit|effee36}}, {{Gerrit|7c15083}}

=== 2019-10-30 ===
* 19:42 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|289a41b}}, {{Gerrit|02a1a4f}}, {{Gerrit|e8d4363}} ([[phab:T224226|T224226]])

=== 2019-10-25 ===
* 15:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|82de2957}} ([[phab:T224212|T224212]], [[phab:T223930|T223930]])
* 14:08 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6a055c2}} ([[phab:T228405|T228405]])
* 13:24 wm-bot: &lt;jeanfred&gt; Deploying from master, will break things.

=== 2019-09-19 ===
* 18:38 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d214f09}}, {{Gerrit|9b8dfa5}}
* 17:27 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|12ebc39}}

=== 2019-07-07 ===
* 22:49 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|5b4a1c6}}, {{Gerrit|ab90227}}, {{Gerrit|d101ca1}}, {{Gerrit|394be6a}}, {{Gerrit|ae16d68}}, {{Gerrit|728a8d2}}

=== 2019-05-30 ===
* 12:08 wm-bot: &lt;jeanfred&gt; Nuked the virtualenv and reinstalled all deps from scratch, in desperation for [[phab:T224651|T224651]]
* 10:59 wm-bot: &lt;jeanfred&gt; Service stop, mv logs, service start for [[phab:T224651|T224651]]

=== 2019-05-22 ===
* 16:39 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8e76a8a}}
* 13:13 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2fdede2}}, {{Gerrit|6b6c07c}}, {{Gerrit|c17cb78}} ([[phab:T224090|T224090]])

=== 2019-05-20 ===
* 19:01 wm-bot: &lt;jeanfred&gt; Restart uwsgi with 10 workers rather than the default of 4
* 19:00 wm-bot: &lt;jeanfred&gt; --help


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>ewiyu4u5e4ds6q9g5s7heiriyl98w8p</sha1>
    </revision>
    <revision>
      <id>2408833</id>
      <parentid>2408832</parentid>
      <timestamp>2026-05-03T19:44:26Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>wmbot~jeanfred@tools-bastion-15: Deploy 9fbf0bd (Add grouping_link_mode config and wire up page creation) for T368001</comment>
      <origin>2408833</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="24112" sha1="2dakzmymqzo57lv9sa9gy5r4v4wpk9b" xml:space="preserve">=== 2026-05-03 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fbf0bd}} (Add grouping_link_mode config and wire up page creation) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|729decc}} (Add GroupingPageCreator to create Listeria subpages) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|74bf18f}} (Add Listeria wikitext formatting to Grouping rows) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9b32a82}} (Split retrieve_and_process_data in process_page) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4dcc1a0}} (Extract prepare_report_groupings from process_data in PropertyStatistics) for [[phab:T368001|T368001]]
* 19:41 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|928f398}} (Remove obsolete version attribute from docker-compose.yml)

=== 2026-05-01 ===
* 18:16 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|01e97d6}} (Extract grouping link strategies into GroupingLink hierarchy), {{Gerrit|359770d}} (Remove pass-through methods for grouping link in GroupingConfiguration), {{Gerrit|54e4a7b}} (Add lang parameter to get_label_for_variable), {{Gerrit|cccd661}} (Simplify SPARQL for mul language in get_label_for_variable), {{Gerrit|548383d}} (Support any label language for grouping links), {{Gerrit|d571596}} (Support QID-based grouping links), {{Gerrit|cfd7c68}} (
* 18:15 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|19b856d}}, {{Gerrit|473fbeb}}, {{Gerrit|0cb0915}}

=== 2026-03-27 ===
* 22:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2bad8b3}} (Catch ModuleNotFoundError when unpickling stale cache entries)

=== 2026-03-24 ===
* 16:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|7883b73}} (Switch from bare imports to relative imports)
* 15:56 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5dbe412}} (Resolve grouping type eagerly in PropertyStatistics constructor)

=== 2026-03-23 ===
* 22:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|1b60d39}} (Handle stale pickle cache entries referencing removed classes)
* 22:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|124f4ca}} (Replace wikibase:isSomeValue with STRSTARTS filter for QLever compatibility)
* 21:28 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fd5828f}} (Detect grouping type via SPARQL and flatten to single GroupingConfiguration)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|e9fae82}} (Add AbstractGroupingType base class for grouping type strategies)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fee0ed5}} (Extract GroupingType strategy classes from GroupingConfiguration hierarchy)

=== 2026-03-18 ===
* 20:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f9dfb79}} (Add /healthz endpoint for automatic health check)

=== 2026-03-17 ===
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2940962}} (Replace GROUP_MAPPING Enum with SPECIAL_GROUPINGS tuple)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|420658b}} (Add MARKER attribute to special grouping classes)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|976599f}} (Rename UnknownValueGrouping.TITLE to MARKER)

=== 2026-03-14 ===
* 16:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be09048}} (Add Wikimedia Commons support using QLever endpoint) for [[phab:T294893|T294893]]
* 16:38 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|ae53e61}} (Derive QLever UI URL from engine endpoint)
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5134f77}} (Use ka/Ma unit suffixes for geological-scale year grouping headings) for [[phab:T236590|T236590]]
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8a735a0}} (Add automatic date resolution rebinning for year groupings) for [[phab:T236590|T236590]]
* 15:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|6478051}}, {{Gerrit|4206c09}}, {{Gerrit|0e920fe}}, {{Gerrit|4eefea1}}

=== 2026-03-13 ===
* 20:33 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a39f1d9}} (Add proper handling for transient Wikidata server errors) for [[phab:T415439|T415439]]
* 20:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|0c418a5}} (Fix tests hitting live Wikidata API (and potential rate-limits)) for [[phab:T420054|T420054]]

=== 2026-03-11 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9665d8f}} (Remove obsolete GroupingType enum from column.py)
* 21:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|60d500a}} (Remove backward-compatible formatting wrapper methods from PropertyStatistics)
* 21:20 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a452c28}} (Extract formatting logic into separate ResultsFormatter class)
* 17:29 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|02338eb}} (Move heading strings to class constants in Grouping objects) and {{Gerrit|d730307}} (Format NoGroup and Totals in  with the other groupings)

=== 2026-03-09 ===
* 18:32 wmbot~jeanfred@tools-bastion-15: Recreated Python 3.11 virtual environment using 'toolforge webservice python3.11 shell' / 'webservice-python-bootstrap --fresh' / '/data/project/integraality/www/python/venv/bin/pip install -r /data/project/integraality/integraality/requirements.txt'
* 18:30 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|55f1132}} (Switch to Python 3.11 as default interpreter)

=== 2026-03-08 ===
* 11:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|37fcc25}} (Extract jobs-framework runtime image as its own variable) and {{Gerrit|1f9bfbb}} (Add service.template deployment to Ansible)

=== 2026-03-06 ===
* 10:08 wmbot~jeanfred@tools-bastion-15: Successfully installed flask-3.1.3 mwparserfromhell-0.7.2 pywikibot-10.7.6 redis-7.0.1 (using a webservice python3.9 shell)
* 10:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fa536f}}, {{Gerrit|9e5111c}}, {{Gerrit|e176f18}}, {{Gerrit|eff3045}}, {{Gerrit|8532e32}} (Dependencies upgrades)

=== 2026-03-05 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5075934}} (Add support for user-specified groupings via template parameter) for [[phab:T419059|T419059]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|3c9fa98}} (Remove two debug print statements)
* 18:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28d800e}} (Extract rdfs:label SPARQL fragment from grouping.py to get_label_for_variable utility), {{Gerrit|40c39de}} (Make intermediate label variables unique to avoid clashes in get_label_for_variable) and {{Gerrit|64e8f94}} (Replace SERVICE wikibase:label with native SPARQL for QLever compatibility) for [[phab:T385749|T385749]]

=== 2025-11-03 ===
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]

=== 2025-10-29 ===
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f40974f}} (Add links to QLever in the Queries) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28c2e1b}} (Add link to QLever on queries error page) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4cc2bcc}} (Add configurable SPARQL endpoint support for QLever in PagesProcessor) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be5c234}} (Add support for QLever as alternative SPARQL query engine) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f34624d}} (Move pywikibot exception handling to SPARQL engine class) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4da9f33}} (Implement SPARQL query engine dependency injection) for [[phab:T385749|T385749]]

=== 2025-10-28 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|92ab962}} (Update pywikibot from 10.0.0 to latest 10.6.0)
* 21:34 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|77deb17}} (Remove link to 'SGE-jobs' tool in footer)

=== 2025-10-27 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8db01c6}} (Simplify Flask webserver startup in app.py)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|31f14b0}} (Bump ruff pre-commit hook from 0.9.3 to latest 0.14.2)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|995121d}} (Bump uv pre-commit hook from 0.5.24 to latest 0.9.5 and regenerate requirements files)
* 09:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|69669c0}} (Optimize 'get_grouping_information_query' query using subquery) for [[phab:T400480|T400480]]

=== 2025-10-25 ===
* 09:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9ac4c1f}} (Convert SPARQL query tests to use triple-quoted strings in PropertyStatisticsTest)

=== 2025-10-24 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5511fd0}} (Fix test_get_grouping_information_query_with_grouping_link for YearGrouping)
* 19:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8bb5d37}} (Convert SPARQL query tests to use triple-quoted strings)
* 19:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|792b38e}} (Remove redundant GROUP BY parameter in 'get_grouping_information_query')

=== 2025-05-06 ===
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|284a70a}} (Update pywikibot from 9.6.1 to latest 10.0.0)
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|1f030a9}} (Re-generate uv lockfile to add 'upload-time' markers)
* 11:19 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8f2391e}} (Replace deprecated  with  in two unit tests)

=== 2025-05-04 ===
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7a2fa93}} (Fallback to mul when formatting grouping links) for [[phab:T393166|T393166]]
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)

=== 2025-05-03 ===
* 19:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|781934d}} (Move grouping-link to Grouping objects and retrieve via SPARQL) for [[phab:T237276|T237276]]

=== 2025-05-02 ===
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5bc6fe9}} (Remove unused local variable in GroupingTest.test unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|f5f0044}} (Remove unused local variable in  unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a742a2}} (Remove unused column_key local variable from get_query_for_items_for_property_[posi{{!}}nega]tive)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|620ac00}} (Remove unused patcher in QueriesTests setUp method)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|e32b9b9}} (Allow to optionally pass wikiproject data to SitelinkColumn constructor)

=== 2025-05-01 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|2a91226}} (Catch WDQS Timeouts and raise proper exception for all SPARQL queries) for [[phab:T278156|T278156]]
* 20:29 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|397c5e8}} (Also catch ServerError to handle WDQS timeouts) for [[phab:T384882|T384882]]

=== 2025-04-28 ===
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|eba498e}} (Use correct namespace when using pywikibot exceptions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ea55660}} (Add unit-tests for page_saving.py functions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|58918e6}} (Tweak linebreak in get_grouping_information_query which was breaking unit tests)

=== 2025-01-23 ===
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|a76ceec}} (Switch from Pipenv to uv as project manager)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5941f3f}} (Upgrade all Python dev-dependencies)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix 'QueriesTests' unit tests in test_app)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix  unit tests in test_app)

=== 2025-01-17 ===
* 10:03 wmbot~jeanfred@tools-sgebastion-10: Recreated virtual environment using 'toolforge webservice python3.9 shell' / 'webservice-python-bootstrap --fresh'

=== 2025-01-16 ===
* 16:36 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|525c9ed}} (Upgrade all Python dependencies)

=== 2024-07-01 ===
* 19:54 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8eecfc6}} &amp; {{Gerrit|0466207}}

=== 2024-06-06 ===
* 12:47 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|022fa2f}} ([[phab:T312727|T312727]])
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ec28a13}}
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7adc07f}}

=== 2024-05-10 ===
* 15:57 wmbot~multichill@tools-bastion-12: Leaving project

=== 2024-05-04 ===
* 20:22 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|51ba697}}
* 20:07 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8ec2fec}} ([[phab:T251008|T251008]])

=== 2024-05-03 ===
* 12:01 wmbot~multichill@tools-bastion-12: emails: all -

=== 2024-04-30 ===
* 19:10 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5a91a78}}, {{Gerrit|864d518}}, c9b199 ([[phab:T251008|T251008]]), {{Gerrit|0dd731f}}

=== 2024-03-17 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a93a59}} ([[phab:T319813|T319813]])
* 20:25 wmbot~jeanfred@tools-sgebastion-10: Re-created virtual environment for Python 3.9 ([[phab:T319813|T319813]])
* 19:26 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|bde5715}}, {{Gerrit|9bdfb02}} ([[phab:T319813|T319813]])

=== 2023-11-21 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|992670c}} ([[phab:T351574|T351574]])
* 19:55 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|1f41a79}}, {{Gerrit|38e7623}}

=== 2023-10-17 ===
* 14:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d7f09b3}}, {{Gerrit|eb1de62}}, {{Gerrit|f4e3656}}, {{Gerrit|82f90c4}} ([[phab:T312726|T312726]])

=== 2023-10-03 ===
* 21:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 19:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 18:04 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|da99820}}, {{Gerrit|801ddeb}}, {{Gerrit|f2ffd22}} ([[phab:T312729|T312729]])

=== 2023-10-02 ===
* 09:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3f9a3d5}}

=== 2023-09-26 ===
* 18:25 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3caa9ed}}, {{Gerrit|a8029ed}}, {{Gerrit|c88ad35}}, {{Gerrit|e575508}}, {{Gerrit|3e3f47f}}, {{Gerrit|63ab297}}, {{Gerrit|06cb52d}}, {{Gerrit|ac64840}}, {{Gerrit|6178698}} ([[phab:T312728|T312728]])

=== 2023-06-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5aecae8}}, {{Gerrit|649259f}}, {{Gerrit|9712666}} ([[phab:T338684|T338684]])

=== 2022-12-26 ===
* 10:48 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c90f9ef}} ([[phab:T325936|T325936]])

=== 2022-09-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|bcbd306}}

=== 2022-07-12 ===
* 15:04 wm-bot: &lt;jeanfred&gt; Triggering full update
* 15:02 wm-bot: &lt;jeanfred&gt; jstop {{Gerrit|5976623}} to kill weekly update job stuck since May 22

=== 2022-06-03 ===
* 12:37 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9e85ede}} ([[phab:T309861|T309861]])

=== 2022-05-15 ===
* 21:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3a675b6}}
* 20:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|80f073e}}
* 19:31 wm-bot: &lt;jeanfred&gt; Uninstall removed requirements and restart webservice
* 19:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|f6dc34b}}
* 19:15 wm-bot: &lt;jeanfred&gt; Install all requirements and restart webservice
* 19:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d395b4c}}, {{Gerrit|591c450}}, {{Gerrit|6d09e5b}}

=== 2022-05-14 ===
* 14:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2a521d1}}, {{Gerrit|29d6b62}}, {{Gerrit|8bd793d}} ([[phab:T257942|T257942]])

=== 2022-05-11 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|43f07d2}}, {{Gerrit|da1f2ef}}, {{Gerrit|6ab7547}}, {{Gerrit|5f6b20a}}, {{Gerrit|0df2121}}

=== 2022-01-28 ===
* 20:44 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|13ed7a3}}

=== 2021-11-01 ===
* 20:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|972a1df}}

=== 2021-10-30 ===
* 19:36 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c1628b9}} and {{Gerrit|b6940b0}}

=== 2021-10-29 ===
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|40c8cd3}}
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|af49bab}} ([[phab:T236590|T236590]] and [[phab:T273226|T273226]])
* 09:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2260c8f}} (follow up to [[phab:T294570|T294570]])
* 09:16 wm-bot: &lt;jeanfred&gt; Install requirements in py37 venv ([[phab:T294570|T294570]])
* 09:14 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a4560a}} and {{Gerrit|de296cd}} ([[phab:T294570|T294570]])

=== 2021-10-28 ===
* 17:14 wm-bot: &lt;jeanfred&gt; Rollbacked to {{Gerrit|977236b42daef66818e561b49e64264a0c785454}} for [[phab:T294570|T294570]] and restart
* 12:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a8e7fb}} ([[phab:T236590|T236590]])
* 11:31 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|30a2f2ba}} ([[phab:T278156|T278156]])
* 11:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5c547d8}}
* 10:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b4f12bc}} to help investigate [[phab:T236960|T236960]]
* 09:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|137f5b8}}, {{Gerrit|c816e41}} {{Gerrit|8e0b9ca}}, {{Gerrit|8dbc099}} ([[phab:T273226|T273226]])

=== 2021-10-27 ===
* 09:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6194d84}} ([[phab:T273226|T273226]])

=== 2021-10-25 ===
* 21:41 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|7e3343a}} ([[phab:T278156|T278156]])
* 20:22 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|ef0f537}} ([[phab:T284684|T284684]])
* 20:15 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|977236b}} ([[phab:T284183|T284183]])
* 18:47 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5dfef39}}, {{Gerrit|e940ae7}}, {{Gerrit|765b314}}, {{Gerrit|3e4c2de}}, {{Gerrit|c859c45}}, {{Gerrit|5526cb3}}

=== 2021-06-11 ===
* 15:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|a753cb2}}
* 15:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5187a4d}}

=== 2021-05-31 ===
* 09:20 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|79e458d}}

=== 2021-05-22 ===
* 20:58 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|258aae9a}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|67c42f43}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d170ea1}}
* 20:24 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2f39621}}
* 20:21 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b52079f}}
* 17:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|8ffbad82c3}} ([[phab:T279236|T279236]])
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6de2308}} and {{Gerrit|c436760}} ([[phab:T279236|T279236]])
* 16:06 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0966be04}} ([[phab:T279236|T279236]])
* 15:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9727a01c}}
* 14:02 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|69048ef}}
* 13:00 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|633eeb0f}}
* 12:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d4fa363}} ([[phab:T283287|T283287]])

=== 2021-04-03 ===
* 21:34 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|666222b}} and {{Gerrit|0dcafff}} ([[phab:T279236|T279236]])
* 21:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])
* 16:45 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])

=== 2021-03-26 ===
* 20:57 wm-bot: &lt;jeanfred&gt; Reinstalled requirements in both virtual environments for [[phab:T240312|T240312]]
* 20:55 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|bcc4fe41}} ([[phab:T240312|T240312]])

=== 2020-07-06 ===
* 22:55 wm-bot: &lt;root&gt; Migrated .webservicerc to service.template ([[phab:T257229|T257229]])

=== 2020-06-18 ===
* 11:01 wm-bot: &lt;jeanfred&gt; Restarted with --canonical for Toolforge domain migration

=== 2020-05-01 ===
* 22:59 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|f0db935}}, {{Gerrit|f8d9fdf}}, {{Gerrit|f57c060}}, {{Gerrit|6c534b7}} ([[phab:T248788|T248788]])

=== 2020-04-29 ===
* 20:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eabbac8}}, {{Gerrit|b74b352}} ([[phab:T248788|T248788]])

=== 2020-04-24 ===
* 22:22 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6963040}}, {{Gerrit|35a3ea7}}, {{Gerrit|22e27b5}}, {{Gerrit|9dd44ff}}, {{Gerrit|0e45815}}, {{Gerrit|6f36ffd}} ([[phab:T248788|T248788]])
* 19:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|9fea19b}}, {{Gerrit|f38047e}}

=== 2020-02-25 ===
* 23:35 wm-bot: &lt;root&gt; Migrated to 2020 Kubernetes cluster

=== 2020-02-16 ===
* 16:49 JeanFred: &quot;Deploy latest from Git master: {{Gerrit|0fcb522}} ([[phab:T243780|T243780]]), {{Gerrit|4cb6a5a}} ([[phab:T243780|T243780]], [[phab:T239067|T239067]])&quot;

=== 2020-02-11 ===
* 21:23 wm-bot157: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|c350e4f}} ([[phab:T243998|T243998]]), {{Gerrit|fa12c21}}

=== 2020-02-05 ===
* 12:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8977958}} ([[phab:T243780|T243780]])

=== 2020-02-03 ===
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|3b2867f}}, {{Gerrit|10f67fd}}, {{Gerrit|2e6a669}} ([[phab:T244030|T244030]]), {{Gerrit|279c92b}}, {{Gerrit|de6308e}}

=== 2020-01-16 ===
* 21:44 wm-bot: &lt;jeanfred&gt; Perform webservice restart for [[phab:T242967|T242967]]

=== 2019-12-11 ===
* 15:15 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|257618f1}} ([[phab:T240312|T240312]])

=== 2019-12-02 ===
* 06:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|770cc93}} ([[phab:T237182|T237182]])

=== 2019-11-25 ===
* 11:36 wm-bot: &lt;jeanfred&gt; Trigger a manual update of all dashboards ([[phab:T239085|T239085]])
* 11:34 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|684fb77}} ([[phab:T239085|T239085]])

=== 2019-11-21 ===
* 08:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2f94256}} ([[phab:T237187|T237187]])

=== 2019-11-10 ===
* 15:31 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|ab714de}} ([[phab:T224226|T224226]])

=== 2019-11-06 ===
* 09:12 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|7202182}} ([[phab:T237187|T237187]])
* 09:01 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|29b19c3}} ([[phab:T237182|T237182]])

=== 2019-11-04 ===
* 17:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d0f4222}} ([[phab:T237189|T237189]])
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|588dae8}}
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|471deddd}} ([[phab:T237182|T237182]])

=== 2019-10-31 ===
* 09:06 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|46484f5}} ([[phab:T224226|T224226]])
* 09:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2d71154}}, {{Gerrit|d0d2937}}, {{Gerrit|905007b}}, {{Gerrit|effee36}}, {{Gerrit|7c15083}}

=== 2019-10-30 ===
* 19:42 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|289a41b}}, {{Gerrit|02a1a4f}}, {{Gerrit|e8d4363}} ([[phab:T224226|T224226]])

=== 2019-10-25 ===
* 15:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|82de2957}} ([[phab:T224212|T224212]], [[phab:T223930|T223930]])
* 14:08 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6a055c2}} ([[phab:T228405|T228405]])
* 13:24 wm-bot: &lt;jeanfred&gt; Deploying from master, will break things.

=== 2019-09-19 ===
* 18:38 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d214f09}}, {{Gerrit|9b8dfa5}}
* 17:27 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|12ebc39}}

=== 2019-07-07 ===
* 22:49 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|5b4a1c6}}, {{Gerrit|ab90227}}, {{Gerrit|d101ca1}}, {{Gerrit|394be6a}}, {{Gerrit|ae16d68}}, {{Gerrit|728a8d2}}

=== 2019-05-30 ===
* 12:08 wm-bot: &lt;jeanfred&gt; Nuked the virtualenv and reinstalled all deps from scratch, in desperation for [[phab:T224651|T224651]]
* 10:59 wm-bot: &lt;jeanfred&gt; Service stop, mv logs, service start for [[phab:T224651|T224651]]

=== 2019-05-22 ===
* 16:39 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8e76a8a}}
* 13:13 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2fdede2}}, {{Gerrit|6b6c07c}}, {{Gerrit|c17cb78}} ([[phab:T224090|T224090]])

=== 2019-05-20 ===
* 19:01 wm-bot: &lt;jeanfred&gt; Restart uwsgi with 10 workers rather than the default of 4
* 19:00 wm-bot: &lt;jeanfred&gt; --help


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>2dakzmymqzo57lv9sa9gy5r4v4wpk9b</sha1>
    </revision>
    <revision>
      <id>2408834</id>
      <parentid>2408833</parentid>
      <timestamp>2026-05-03T19:44:33Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>wmbot~jeanfred@tools-bastion-15: Deploy b41db93 (Re-resolve grouping links after year rebinning) for T368001</comment>
      <origin>2408834</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="24257" sha1="nrnagu2zhiua75x55oxff02dj26eca1" xml:space="preserve">=== 2026-05-03 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|b41db93}} (Re-resolve grouping links after year rebinning) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fbf0bd}} (Add grouping_link_mode config and wire up page creation) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|729decc}} (Add GroupingPageCreator to create Listeria subpages) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|74bf18f}} (Add Listeria wikitext formatting to Grouping rows) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9b32a82}} (Split retrieve_and_process_data in process_page) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4dcc1a0}} (Extract prepare_report_groupings from process_data in PropertyStatistics) for [[phab:T368001|T368001]]
* 19:41 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|928f398}} (Remove obsolete version attribute from docker-compose.yml)

=== 2026-05-01 ===
* 18:16 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|01e97d6}} (Extract grouping link strategies into GroupingLink hierarchy), {{Gerrit|359770d}} (Remove pass-through methods for grouping link in GroupingConfiguration), {{Gerrit|54e4a7b}} (Add lang parameter to get_label_for_variable), {{Gerrit|cccd661}} (Simplify SPARQL for mul language in get_label_for_variable), {{Gerrit|548383d}} (Support any label language for grouping links), {{Gerrit|d571596}} (Support QID-based grouping links), {{Gerrit|cfd7c68}} (
* 18:15 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|19b856d}}, {{Gerrit|473fbeb}}, {{Gerrit|0cb0915}}

=== 2026-03-27 ===
* 22:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2bad8b3}} (Catch ModuleNotFoundError when unpickling stale cache entries)

=== 2026-03-24 ===
* 16:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|7883b73}} (Switch from bare imports to relative imports)
* 15:56 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5dbe412}} (Resolve grouping type eagerly in PropertyStatistics constructor)

=== 2026-03-23 ===
* 22:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|1b60d39}} (Handle stale pickle cache entries referencing removed classes)
* 22:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|124f4ca}} (Replace wikibase:isSomeValue with STRSTARTS filter for QLever compatibility)
* 21:28 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fd5828f}} (Detect grouping type via SPARQL and flatten to single GroupingConfiguration)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|e9fae82}} (Add AbstractGroupingType base class for grouping type strategies)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fee0ed5}} (Extract GroupingType strategy classes from GroupingConfiguration hierarchy)

=== 2026-03-18 ===
* 20:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f9dfb79}} (Add /healthz endpoint for automatic health check)

=== 2026-03-17 ===
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2940962}} (Replace GROUP_MAPPING Enum with SPECIAL_GROUPINGS tuple)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|420658b}} (Add MARKER attribute to special grouping classes)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|976599f}} (Rename UnknownValueGrouping.TITLE to MARKER)

=== 2026-03-14 ===
* 16:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be09048}} (Add Wikimedia Commons support using QLever endpoint) for [[phab:T294893|T294893]]
* 16:38 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|ae53e61}} (Derive QLever UI URL from engine endpoint)
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5134f77}} (Use ka/Ma unit suffixes for geological-scale year grouping headings) for [[phab:T236590|T236590]]
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8a735a0}} (Add automatic date resolution rebinning for year groupings) for [[phab:T236590|T236590]]
* 15:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|6478051}}, {{Gerrit|4206c09}}, {{Gerrit|0e920fe}}, {{Gerrit|4eefea1}}

=== 2026-03-13 ===
* 20:33 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a39f1d9}} (Add proper handling for transient Wikidata server errors) for [[phab:T415439|T415439]]
* 20:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|0c418a5}} (Fix tests hitting live Wikidata API (and potential rate-limits)) for [[phab:T420054|T420054]]

=== 2026-03-11 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9665d8f}} (Remove obsolete GroupingType enum from column.py)
* 21:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|60d500a}} (Remove backward-compatible formatting wrapper methods from PropertyStatistics)
* 21:20 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a452c28}} (Extract formatting logic into separate ResultsFormatter class)
* 17:29 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|02338eb}} (Move heading strings to class constants in Grouping objects) and {{Gerrit|d730307}} (Format NoGroup and Totals in  with the other groupings)

=== 2026-03-09 ===
* 18:32 wmbot~jeanfred@tools-bastion-15: Recreated Python 3.11 virtual environment using 'toolforge webservice python3.11 shell' / 'webservice-python-bootstrap --fresh' / '/data/project/integraality/www/python/venv/bin/pip install -r /data/project/integraality/integraality/requirements.txt'
* 18:30 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|55f1132}} (Switch to Python 3.11 as default interpreter)

=== 2026-03-08 ===
* 11:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|37fcc25}} (Extract jobs-framework runtime image as its own variable) and {{Gerrit|1f9bfbb}} (Add service.template deployment to Ansible)

=== 2026-03-06 ===
* 10:08 wmbot~jeanfred@tools-bastion-15: Successfully installed flask-3.1.3 mwparserfromhell-0.7.2 pywikibot-10.7.6 redis-7.0.1 (using a webservice python3.9 shell)
* 10:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fa536f}}, {{Gerrit|9e5111c}}, {{Gerrit|e176f18}}, {{Gerrit|eff3045}}, {{Gerrit|8532e32}} (Dependencies upgrades)

=== 2026-03-05 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5075934}} (Add support for user-specified groupings via template parameter) for [[phab:T419059|T419059]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|3c9fa98}} (Remove two debug print statements)
* 18:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28d800e}} (Extract rdfs:label SPARQL fragment from grouping.py to get_label_for_variable utility), {{Gerrit|40c39de}} (Make intermediate label variables unique to avoid clashes in get_label_for_variable) and {{Gerrit|64e8f94}} (Replace SERVICE wikibase:label with native SPARQL for QLever compatibility) for [[phab:T385749|T385749]]

=== 2025-11-03 ===
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]

=== 2025-10-29 ===
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f40974f}} (Add links to QLever in the Queries) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28c2e1b}} (Add link to QLever on queries error page) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4cc2bcc}} (Add configurable SPARQL endpoint support for QLever in PagesProcessor) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be5c234}} (Add support for QLever as alternative SPARQL query engine) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f34624d}} (Move pywikibot exception handling to SPARQL engine class) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4da9f33}} (Implement SPARQL query engine dependency injection) for [[phab:T385749|T385749]]

=== 2025-10-28 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|92ab962}} (Update pywikibot from 10.0.0 to latest 10.6.0)
* 21:34 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|77deb17}} (Remove link to 'SGE-jobs' tool in footer)

=== 2025-10-27 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8db01c6}} (Simplify Flask webserver startup in app.py)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|31f14b0}} (Bump ruff pre-commit hook from 0.9.3 to latest 0.14.2)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|995121d}} (Bump uv pre-commit hook from 0.5.24 to latest 0.9.5 and regenerate requirements files)
* 09:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|69669c0}} (Optimize 'get_grouping_information_query' query using subquery) for [[phab:T400480|T400480]]

=== 2025-10-25 ===
* 09:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9ac4c1f}} (Convert SPARQL query tests to use triple-quoted strings in PropertyStatisticsTest)

=== 2025-10-24 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5511fd0}} (Fix test_get_grouping_information_query_with_grouping_link for YearGrouping)
* 19:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8bb5d37}} (Convert SPARQL query tests to use triple-quoted strings)
* 19:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|792b38e}} (Remove redundant GROUP BY parameter in 'get_grouping_information_query')

=== 2025-05-06 ===
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|284a70a}} (Update pywikibot from 9.6.1 to latest 10.0.0)
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|1f030a9}} (Re-generate uv lockfile to add 'upload-time' markers)
* 11:19 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8f2391e}} (Replace deprecated  with  in two unit tests)

=== 2025-05-04 ===
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7a2fa93}} (Fallback to mul when formatting grouping links) for [[phab:T393166|T393166]]
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)

=== 2025-05-03 ===
* 19:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|781934d}} (Move grouping-link to Grouping objects and retrieve via SPARQL) for [[phab:T237276|T237276]]

=== 2025-05-02 ===
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5bc6fe9}} (Remove unused local variable in GroupingTest.test unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|f5f0044}} (Remove unused local variable in  unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a742a2}} (Remove unused column_key local variable from get_query_for_items_for_property_[posi{{!}}nega]tive)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|620ac00}} (Remove unused patcher in QueriesTests setUp method)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|e32b9b9}} (Allow to optionally pass wikiproject data to SitelinkColumn constructor)

=== 2025-05-01 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|2a91226}} (Catch WDQS Timeouts and raise proper exception for all SPARQL queries) for [[phab:T278156|T278156]]
* 20:29 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|397c5e8}} (Also catch ServerError to handle WDQS timeouts) for [[phab:T384882|T384882]]

=== 2025-04-28 ===
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|eba498e}} (Use correct namespace when using pywikibot exceptions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ea55660}} (Add unit-tests for page_saving.py functions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|58918e6}} (Tweak linebreak in get_grouping_information_query which was breaking unit tests)

=== 2025-01-23 ===
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|a76ceec}} (Switch from Pipenv to uv as project manager)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5941f3f}} (Upgrade all Python dev-dependencies)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix 'QueriesTests' unit tests in test_app)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix  unit tests in test_app)

=== 2025-01-17 ===
* 10:03 wmbot~jeanfred@tools-sgebastion-10: Recreated virtual environment using 'toolforge webservice python3.9 shell' / 'webservice-python-bootstrap --fresh'

=== 2025-01-16 ===
* 16:36 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|525c9ed}} (Upgrade all Python dependencies)

=== 2024-07-01 ===
* 19:54 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8eecfc6}} &amp; {{Gerrit|0466207}}

=== 2024-06-06 ===
* 12:47 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|022fa2f}} ([[phab:T312727|T312727]])
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ec28a13}}
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7adc07f}}

=== 2024-05-10 ===
* 15:57 wmbot~multichill@tools-bastion-12: Leaving project

=== 2024-05-04 ===
* 20:22 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|51ba697}}
* 20:07 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8ec2fec}} ([[phab:T251008|T251008]])

=== 2024-05-03 ===
* 12:01 wmbot~multichill@tools-bastion-12: emails: all -

=== 2024-04-30 ===
* 19:10 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5a91a78}}, {{Gerrit|864d518}}, c9b199 ([[phab:T251008|T251008]]), {{Gerrit|0dd731f}}

=== 2024-03-17 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a93a59}} ([[phab:T319813|T319813]])
* 20:25 wmbot~jeanfred@tools-sgebastion-10: Re-created virtual environment for Python 3.9 ([[phab:T319813|T319813]])
* 19:26 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|bde5715}}, {{Gerrit|9bdfb02}} ([[phab:T319813|T319813]])

=== 2023-11-21 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|992670c}} ([[phab:T351574|T351574]])
* 19:55 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|1f41a79}}, {{Gerrit|38e7623}}

=== 2023-10-17 ===
* 14:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d7f09b3}}, {{Gerrit|eb1de62}}, {{Gerrit|f4e3656}}, {{Gerrit|82f90c4}} ([[phab:T312726|T312726]])

=== 2023-10-03 ===
* 21:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 19:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 18:04 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|da99820}}, {{Gerrit|801ddeb}}, {{Gerrit|f2ffd22}} ([[phab:T312729|T312729]])

=== 2023-10-02 ===
* 09:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3f9a3d5}}

=== 2023-09-26 ===
* 18:25 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3caa9ed}}, {{Gerrit|a8029ed}}, {{Gerrit|c88ad35}}, {{Gerrit|e575508}}, {{Gerrit|3e3f47f}}, {{Gerrit|63ab297}}, {{Gerrit|06cb52d}}, {{Gerrit|ac64840}}, {{Gerrit|6178698}} ([[phab:T312728|T312728]])

=== 2023-06-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5aecae8}}, {{Gerrit|649259f}}, {{Gerrit|9712666}} ([[phab:T338684|T338684]])

=== 2022-12-26 ===
* 10:48 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c90f9ef}} ([[phab:T325936|T325936]])

=== 2022-09-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|bcbd306}}

=== 2022-07-12 ===
* 15:04 wm-bot: &lt;jeanfred&gt; Triggering full update
* 15:02 wm-bot: &lt;jeanfred&gt; jstop {{Gerrit|5976623}} to kill weekly update job stuck since May 22

=== 2022-06-03 ===
* 12:37 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9e85ede}} ([[phab:T309861|T309861]])

=== 2022-05-15 ===
* 21:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3a675b6}}
* 20:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|80f073e}}
* 19:31 wm-bot: &lt;jeanfred&gt; Uninstall removed requirements and restart webservice
* 19:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|f6dc34b}}
* 19:15 wm-bot: &lt;jeanfred&gt; Install all requirements and restart webservice
* 19:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d395b4c}}, {{Gerrit|591c450}}, {{Gerrit|6d09e5b}}

=== 2022-05-14 ===
* 14:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2a521d1}}, {{Gerrit|29d6b62}}, {{Gerrit|8bd793d}} ([[phab:T257942|T257942]])

=== 2022-05-11 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|43f07d2}}, {{Gerrit|da1f2ef}}, {{Gerrit|6ab7547}}, {{Gerrit|5f6b20a}}, {{Gerrit|0df2121}}

=== 2022-01-28 ===
* 20:44 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|13ed7a3}}

=== 2021-11-01 ===
* 20:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|972a1df}}

=== 2021-10-30 ===
* 19:36 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c1628b9}} and {{Gerrit|b6940b0}}

=== 2021-10-29 ===
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|40c8cd3}}
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|af49bab}} ([[phab:T236590|T236590]] and [[phab:T273226|T273226]])
* 09:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2260c8f}} (follow up to [[phab:T294570|T294570]])
* 09:16 wm-bot: &lt;jeanfred&gt; Install requirements in py37 venv ([[phab:T294570|T294570]])
* 09:14 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a4560a}} and {{Gerrit|de296cd}} ([[phab:T294570|T294570]])

=== 2021-10-28 ===
* 17:14 wm-bot: &lt;jeanfred&gt; Rollbacked to {{Gerrit|977236b42daef66818e561b49e64264a0c785454}} for [[phab:T294570|T294570]] and restart
* 12:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a8e7fb}} ([[phab:T236590|T236590]])
* 11:31 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|30a2f2ba}} ([[phab:T278156|T278156]])
* 11:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5c547d8}}
* 10:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b4f12bc}} to help investigate [[phab:T236960|T236960]]
* 09:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|137f5b8}}, {{Gerrit|c816e41}} {{Gerrit|8e0b9ca}}, {{Gerrit|8dbc099}} ([[phab:T273226|T273226]])

=== 2021-10-27 ===
* 09:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6194d84}} ([[phab:T273226|T273226]])

=== 2021-10-25 ===
* 21:41 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|7e3343a}} ([[phab:T278156|T278156]])
* 20:22 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|ef0f537}} ([[phab:T284684|T284684]])
* 20:15 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|977236b}} ([[phab:T284183|T284183]])
* 18:47 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5dfef39}}, {{Gerrit|e940ae7}}, {{Gerrit|765b314}}, {{Gerrit|3e4c2de}}, {{Gerrit|c859c45}}, {{Gerrit|5526cb3}}

=== 2021-06-11 ===
* 15:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|a753cb2}}
* 15:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5187a4d}}

=== 2021-05-31 ===
* 09:20 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|79e458d}}

=== 2021-05-22 ===
* 20:58 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|258aae9a}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|67c42f43}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d170ea1}}
* 20:24 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2f39621}}
* 20:21 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b52079f}}
* 17:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|8ffbad82c3}} ([[phab:T279236|T279236]])
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6de2308}} and {{Gerrit|c436760}} ([[phab:T279236|T279236]])
* 16:06 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0966be04}} ([[phab:T279236|T279236]])
* 15:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9727a01c}}
* 14:02 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|69048ef}}
* 13:00 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|633eeb0f}}
* 12:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d4fa363}} ([[phab:T283287|T283287]])

=== 2021-04-03 ===
* 21:34 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|666222b}} and {{Gerrit|0dcafff}} ([[phab:T279236|T279236]])
* 21:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])
* 16:45 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])

=== 2021-03-26 ===
* 20:57 wm-bot: &lt;jeanfred&gt; Reinstalled requirements in both virtual environments for [[phab:T240312|T240312]]
* 20:55 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|bcc4fe41}} ([[phab:T240312|T240312]])

=== 2020-07-06 ===
* 22:55 wm-bot: &lt;root&gt; Migrated .webservicerc to service.template ([[phab:T257229|T257229]])

=== 2020-06-18 ===
* 11:01 wm-bot: &lt;jeanfred&gt; Restarted with --canonical for Toolforge domain migration

=== 2020-05-01 ===
* 22:59 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|f0db935}}, {{Gerrit|f8d9fdf}}, {{Gerrit|f57c060}}, {{Gerrit|6c534b7}} ([[phab:T248788|T248788]])

=== 2020-04-29 ===
* 20:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eabbac8}}, {{Gerrit|b74b352}} ([[phab:T248788|T248788]])

=== 2020-04-24 ===
* 22:22 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6963040}}, {{Gerrit|35a3ea7}}, {{Gerrit|22e27b5}}, {{Gerrit|9dd44ff}}, {{Gerrit|0e45815}}, {{Gerrit|6f36ffd}} ([[phab:T248788|T248788]])
* 19:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|9fea19b}}, {{Gerrit|f38047e}}

=== 2020-02-25 ===
* 23:35 wm-bot: &lt;root&gt; Migrated to 2020 Kubernetes cluster

=== 2020-02-16 ===
* 16:49 JeanFred: &quot;Deploy latest from Git master: {{Gerrit|0fcb522}} ([[phab:T243780|T243780]]), {{Gerrit|4cb6a5a}} ([[phab:T243780|T243780]], [[phab:T239067|T239067]])&quot;

=== 2020-02-11 ===
* 21:23 wm-bot157: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|c350e4f}} ([[phab:T243998|T243998]]), {{Gerrit|fa12c21}}

=== 2020-02-05 ===
* 12:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8977958}} ([[phab:T243780|T243780]])

=== 2020-02-03 ===
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|3b2867f}}, {{Gerrit|10f67fd}}, {{Gerrit|2e6a669}} ([[phab:T244030|T244030]]), {{Gerrit|279c92b}}, {{Gerrit|de6308e}}

=== 2020-01-16 ===
* 21:44 wm-bot: &lt;jeanfred&gt; Perform webservice restart for [[phab:T242967|T242967]]

=== 2019-12-11 ===
* 15:15 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|257618f1}} ([[phab:T240312|T240312]])

=== 2019-12-02 ===
* 06:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|770cc93}} ([[phab:T237182|T237182]])

=== 2019-11-25 ===
* 11:36 wm-bot: &lt;jeanfred&gt; Trigger a manual update of all dashboards ([[phab:T239085|T239085]])
* 11:34 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|684fb77}} ([[phab:T239085|T239085]])

=== 2019-11-21 ===
* 08:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2f94256}} ([[phab:T237187|T237187]])

=== 2019-11-10 ===
* 15:31 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|ab714de}} ([[phab:T224226|T224226]])

=== 2019-11-06 ===
* 09:12 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|7202182}} ([[phab:T237187|T237187]])
* 09:01 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|29b19c3}} ([[phab:T237182|T237182]])

=== 2019-11-04 ===
* 17:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d0f4222}} ([[phab:T237189|T237189]])
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|588dae8}}
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|471deddd}} ([[phab:T237182|T237182]])

=== 2019-10-31 ===
* 09:06 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|46484f5}} ([[phab:T224226|T224226]])
* 09:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2d71154}}, {{Gerrit|d0d2937}}, {{Gerrit|905007b}}, {{Gerrit|effee36}}, {{Gerrit|7c15083}}

=== 2019-10-30 ===
* 19:42 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|289a41b}}, {{Gerrit|02a1a4f}}, {{Gerrit|e8d4363}} ([[phab:T224226|T224226]])

=== 2019-10-25 ===
* 15:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|82de2957}} ([[phab:T224212|T224212]], [[phab:T223930|T223930]])
* 14:08 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6a055c2}} ([[phab:T228405|T228405]])
* 13:24 wm-bot: &lt;jeanfred&gt; Deploying from master, will break things.

=== 2019-09-19 ===
* 18:38 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d214f09}}, {{Gerrit|9b8dfa5}}
* 17:27 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|12ebc39}}

=== 2019-07-07 ===
* 22:49 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|5b4a1c6}}, {{Gerrit|ab90227}}, {{Gerrit|d101ca1}}, {{Gerrit|394be6a}}, {{Gerrit|ae16d68}}, {{Gerrit|728a8d2}}

=== 2019-05-30 ===
* 12:08 wm-bot: &lt;jeanfred&gt; Nuked the virtualenv and reinstalled all deps from scratch, in desperation for [[phab:T224651|T224651]]
* 10:59 wm-bot: &lt;jeanfred&gt; Service stop, mv logs, service start for [[phab:T224651|T224651]]

=== 2019-05-22 ===
* 16:39 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8e76a8a}}
* 13:13 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2fdede2}}, {{Gerrit|6b6c07c}}, {{Gerrit|c17cb78}} ([[phab:T224090|T224090]])

=== 2019-05-20 ===
* 19:01 wm-bot: &lt;jeanfred&gt; Restart uwsgi with 10 workers rather than the default of 4
* 19:00 wm-bot: &lt;jeanfred&gt; --help


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>nrnagu2zhiua75x55oxff02dj26eca1</sha1>
    </revision>
    <revision>
      <id>2408835</id>
      <parentid>2408834</parentid>
      <timestamp>2026-05-03T19:44:45Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>wmbot~jeanfred@tools-bastion-15: Deploy 18813aa (Put label and description columns first in Listeria output) for T368001</comment>
      <origin>2408835</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="24414" sha1="r0lvux45gut6xxew10adak7ho0c6hzd" xml:space="preserve">=== 2026-05-03 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|18813aa}} (Put label and description columns first in Listeria output) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|b41db93}} (Re-resolve grouping links after year rebinning) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fbf0bd}} (Add grouping_link_mode config and wire up page creation) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|729decc}} (Add GroupingPageCreator to create Listeria subpages) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|74bf18f}} (Add Listeria wikitext formatting to Grouping rows) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9b32a82}} (Split retrieve_and_process_data in process_page) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4dcc1a0}} (Extract prepare_report_groupings from process_data in PropertyStatistics) for [[phab:T368001|T368001]]
* 19:41 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|928f398}} (Remove obsolete version attribute from docker-compose.yml)

=== 2026-05-01 ===
* 18:16 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|01e97d6}} (Extract grouping link strategies into GroupingLink hierarchy), {{Gerrit|359770d}} (Remove pass-through methods for grouping link in GroupingConfiguration), {{Gerrit|54e4a7b}} (Add lang parameter to get_label_for_variable), {{Gerrit|cccd661}} (Simplify SPARQL for mul language in get_label_for_variable), {{Gerrit|548383d}} (Support any label language for grouping links), {{Gerrit|d571596}} (Support QID-based grouping links), {{Gerrit|cfd7c68}} (
* 18:15 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|19b856d}}, {{Gerrit|473fbeb}}, {{Gerrit|0cb0915}}

=== 2026-03-27 ===
* 22:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2bad8b3}} (Catch ModuleNotFoundError when unpickling stale cache entries)

=== 2026-03-24 ===
* 16:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|7883b73}} (Switch from bare imports to relative imports)
* 15:56 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5dbe412}} (Resolve grouping type eagerly in PropertyStatistics constructor)

=== 2026-03-23 ===
* 22:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|1b60d39}} (Handle stale pickle cache entries referencing removed classes)
* 22:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|124f4ca}} (Replace wikibase:isSomeValue with STRSTARTS filter for QLever compatibility)
* 21:28 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fd5828f}} (Detect grouping type via SPARQL and flatten to single GroupingConfiguration)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|e9fae82}} (Add AbstractGroupingType base class for grouping type strategies)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fee0ed5}} (Extract GroupingType strategy classes from GroupingConfiguration hierarchy)

=== 2026-03-18 ===
* 20:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f9dfb79}} (Add /healthz endpoint for automatic health check)

=== 2026-03-17 ===
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2940962}} (Replace GROUP_MAPPING Enum with SPECIAL_GROUPINGS tuple)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|420658b}} (Add MARKER attribute to special grouping classes)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|976599f}} (Rename UnknownValueGrouping.TITLE to MARKER)

=== 2026-03-14 ===
* 16:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be09048}} (Add Wikimedia Commons support using QLever endpoint) for [[phab:T294893|T294893]]
* 16:38 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|ae53e61}} (Derive QLever UI URL from engine endpoint)
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5134f77}} (Use ka/Ma unit suffixes for geological-scale year grouping headings) for [[phab:T236590|T236590]]
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8a735a0}} (Add automatic date resolution rebinning for year groupings) for [[phab:T236590|T236590]]
* 15:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|6478051}}, {{Gerrit|4206c09}}, {{Gerrit|0e920fe}}, {{Gerrit|4eefea1}}

=== 2026-03-13 ===
* 20:33 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a39f1d9}} (Add proper handling for transient Wikidata server errors) for [[phab:T415439|T415439]]
* 20:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|0c418a5}} (Fix tests hitting live Wikidata API (and potential rate-limits)) for [[phab:T420054|T420054]]

=== 2026-03-11 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9665d8f}} (Remove obsolete GroupingType enum from column.py)
* 21:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|60d500a}} (Remove backward-compatible formatting wrapper methods from PropertyStatistics)
* 21:20 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a452c28}} (Extract formatting logic into separate ResultsFormatter class)
* 17:29 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|02338eb}} (Move heading strings to class constants in Grouping objects) and {{Gerrit|d730307}} (Format NoGroup and Totals in  with the other groupings)

=== 2026-03-09 ===
* 18:32 wmbot~jeanfred@tools-bastion-15: Recreated Python 3.11 virtual environment using 'toolforge webservice python3.11 shell' / 'webservice-python-bootstrap --fresh' / '/data/project/integraality/www/python/venv/bin/pip install -r /data/project/integraality/integraality/requirements.txt'
* 18:30 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|55f1132}} (Switch to Python 3.11 as default interpreter)

=== 2026-03-08 ===
* 11:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|37fcc25}} (Extract jobs-framework runtime image as its own variable) and {{Gerrit|1f9bfbb}} (Add service.template deployment to Ansible)

=== 2026-03-06 ===
* 10:08 wmbot~jeanfred@tools-bastion-15: Successfully installed flask-3.1.3 mwparserfromhell-0.7.2 pywikibot-10.7.6 redis-7.0.1 (using a webservice python3.9 shell)
* 10:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fa536f}}, {{Gerrit|9e5111c}}, {{Gerrit|e176f18}}, {{Gerrit|eff3045}}, {{Gerrit|8532e32}} (Dependencies upgrades)

=== 2026-03-05 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5075934}} (Add support for user-specified groupings via template parameter) for [[phab:T419059|T419059]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|3c9fa98}} (Remove two debug print statements)
* 18:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28d800e}} (Extract rdfs:label SPARQL fragment from grouping.py to get_label_for_variable utility), {{Gerrit|40c39de}} (Make intermediate label variables unique to avoid clashes in get_label_for_variable) and {{Gerrit|64e8f94}} (Replace SERVICE wikibase:label with native SPARQL for QLever compatibility) for [[phab:T385749|T385749]]

=== 2025-11-03 ===
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]

=== 2025-10-29 ===
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f40974f}} (Add links to QLever in the Queries) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28c2e1b}} (Add link to QLever on queries error page) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4cc2bcc}} (Add configurable SPARQL endpoint support for QLever in PagesProcessor) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be5c234}} (Add support for QLever as alternative SPARQL query engine) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f34624d}} (Move pywikibot exception handling to SPARQL engine class) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4da9f33}} (Implement SPARQL query engine dependency injection) for [[phab:T385749|T385749]]

=== 2025-10-28 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|92ab962}} (Update pywikibot from 10.0.0 to latest 10.6.0)
* 21:34 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|77deb17}} (Remove link to 'SGE-jobs' tool in footer)

=== 2025-10-27 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8db01c6}} (Simplify Flask webserver startup in app.py)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|31f14b0}} (Bump ruff pre-commit hook from 0.9.3 to latest 0.14.2)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|995121d}} (Bump uv pre-commit hook from 0.5.24 to latest 0.9.5 and regenerate requirements files)
* 09:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|69669c0}} (Optimize 'get_grouping_information_query' query using subquery) for [[phab:T400480|T400480]]

=== 2025-10-25 ===
* 09:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9ac4c1f}} (Convert SPARQL query tests to use triple-quoted strings in PropertyStatisticsTest)

=== 2025-10-24 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5511fd0}} (Fix test_get_grouping_information_query_with_grouping_link for YearGrouping)
* 19:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8bb5d37}} (Convert SPARQL query tests to use triple-quoted strings)
* 19:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|792b38e}} (Remove redundant GROUP BY parameter in 'get_grouping_information_query')

=== 2025-05-06 ===
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|284a70a}} (Update pywikibot from 9.6.1 to latest 10.0.0)
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|1f030a9}} (Re-generate uv lockfile to add 'upload-time' markers)
* 11:19 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8f2391e}} (Replace deprecated  with  in two unit tests)

=== 2025-05-04 ===
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7a2fa93}} (Fallback to mul when formatting grouping links) for [[phab:T393166|T393166]]
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)

=== 2025-05-03 ===
* 19:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|781934d}} (Move grouping-link to Grouping objects and retrieve via SPARQL) for [[phab:T237276|T237276]]

=== 2025-05-02 ===
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5bc6fe9}} (Remove unused local variable in GroupingTest.test unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|f5f0044}} (Remove unused local variable in  unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a742a2}} (Remove unused column_key local variable from get_query_for_items_for_property_[posi{{!}}nega]tive)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|620ac00}} (Remove unused patcher in QueriesTests setUp method)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|e32b9b9}} (Allow to optionally pass wikiproject data to SitelinkColumn constructor)

=== 2025-05-01 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|2a91226}} (Catch WDQS Timeouts and raise proper exception for all SPARQL queries) for [[phab:T278156|T278156]]
* 20:29 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|397c5e8}} (Also catch ServerError to handle WDQS timeouts) for [[phab:T384882|T384882]]

=== 2025-04-28 ===
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|eba498e}} (Use correct namespace when using pywikibot exceptions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ea55660}} (Add unit-tests for page_saving.py functions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|58918e6}} (Tweak linebreak in get_grouping_information_query which was breaking unit tests)

=== 2025-01-23 ===
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|a76ceec}} (Switch from Pipenv to uv as project manager)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5941f3f}} (Upgrade all Python dev-dependencies)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix 'QueriesTests' unit tests in test_app)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix  unit tests in test_app)

=== 2025-01-17 ===
* 10:03 wmbot~jeanfred@tools-sgebastion-10: Recreated virtual environment using 'toolforge webservice python3.9 shell' / 'webservice-python-bootstrap --fresh'

=== 2025-01-16 ===
* 16:36 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|525c9ed}} (Upgrade all Python dependencies)

=== 2024-07-01 ===
* 19:54 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8eecfc6}} &amp; {{Gerrit|0466207}}

=== 2024-06-06 ===
* 12:47 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|022fa2f}} ([[phab:T312727|T312727]])
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ec28a13}}
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7adc07f}}

=== 2024-05-10 ===
* 15:57 wmbot~multichill@tools-bastion-12: Leaving project

=== 2024-05-04 ===
* 20:22 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|51ba697}}
* 20:07 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8ec2fec}} ([[phab:T251008|T251008]])

=== 2024-05-03 ===
* 12:01 wmbot~multichill@tools-bastion-12: emails: all -

=== 2024-04-30 ===
* 19:10 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5a91a78}}, {{Gerrit|864d518}}, c9b199 ([[phab:T251008|T251008]]), {{Gerrit|0dd731f}}

=== 2024-03-17 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a93a59}} ([[phab:T319813|T319813]])
* 20:25 wmbot~jeanfred@tools-sgebastion-10: Re-created virtual environment for Python 3.9 ([[phab:T319813|T319813]])
* 19:26 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|bde5715}}, {{Gerrit|9bdfb02}} ([[phab:T319813|T319813]])

=== 2023-11-21 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|992670c}} ([[phab:T351574|T351574]])
* 19:55 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|1f41a79}}, {{Gerrit|38e7623}}

=== 2023-10-17 ===
* 14:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d7f09b3}}, {{Gerrit|eb1de62}}, {{Gerrit|f4e3656}}, {{Gerrit|82f90c4}} ([[phab:T312726|T312726]])

=== 2023-10-03 ===
* 21:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 19:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 18:04 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|da99820}}, {{Gerrit|801ddeb}}, {{Gerrit|f2ffd22}} ([[phab:T312729|T312729]])

=== 2023-10-02 ===
* 09:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3f9a3d5}}

=== 2023-09-26 ===
* 18:25 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3caa9ed}}, {{Gerrit|a8029ed}}, {{Gerrit|c88ad35}}, {{Gerrit|e575508}}, {{Gerrit|3e3f47f}}, {{Gerrit|63ab297}}, {{Gerrit|06cb52d}}, {{Gerrit|ac64840}}, {{Gerrit|6178698}} ([[phab:T312728|T312728]])

=== 2023-06-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5aecae8}}, {{Gerrit|649259f}}, {{Gerrit|9712666}} ([[phab:T338684|T338684]])

=== 2022-12-26 ===
* 10:48 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c90f9ef}} ([[phab:T325936|T325936]])

=== 2022-09-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|bcbd306}}

=== 2022-07-12 ===
* 15:04 wm-bot: &lt;jeanfred&gt; Triggering full update
* 15:02 wm-bot: &lt;jeanfred&gt; jstop {{Gerrit|5976623}} to kill weekly update job stuck since May 22

=== 2022-06-03 ===
* 12:37 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9e85ede}} ([[phab:T309861|T309861]])

=== 2022-05-15 ===
* 21:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3a675b6}}
* 20:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|80f073e}}
* 19:31 wm-bot: &lt;jeanfred&gt; Uninstall removed requirements and restart webservice
* 19:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|f6dc34b}}
* 19:15 wm-bot: &lt;jeanfred&gt; Install all requirements and restart webservice
* 19:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d395b4c}}, {{Gerrit|591c450}}, {{Gerrit|6d09e5b}}

=== 2022-05-14 ===
* 14:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2a521d1}}, {{Gerrit|29d6b62}}, {{Gerrit|8bd793d}} ([[phab:T257942|T257942]])

=== 2022-05-11 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|43f07d2}}, {{Gerrit|da1f2ef}}, {{Gerrit|6ab7547}}, {{Gerrit|5f6b20a}}, {{Gerrit|0df2121}}

=== 2022-01-28 ===
* 20:44 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|13ed7a3}}

=== 2021-11-01 ===
* 20:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|972a1df}}

=== 2021-10-30 ===
* 19:36 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c1628b9}} and {{Gerrit|b6940b0}}

=== 2021-10-29 ===
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|40c8cd3}}
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|af49bab}} ([[phab:T236590|T236590]] and [[phab:T273226|T273226]])
* 09:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2260c8f}} (follow up to [[phab:T294570|T294570]])
* 09:16 wm-bot: &lt;jeanfred&gt; Install requirements in py37 venv ([[phab:T294570|T294570]])
* 09:14 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a4560a}} and {{Gerrit|de296cd}} ([[phab:T294570|T294570]])

=== 2021-10-28 ===
* 17:14 wm-bot: &lt;jeanfred&gt; Rollbacked to {{Gerrit|977236b42daef66818e561b49e64264a0c785454}} for [[phab:T294570|T294570]] and restart
* 12:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a8e7fb}} ([[phab:T236590|T236590]])
* 11:31 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|30a2f2ba}} ([[phab:T278156|T278156]])
* 11:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5c547d8}}
* 10:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b4f12bc}} to help investigate [[phab:T236960|T236960]]
* 09:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|137f5b8}}, {{Gerrit|c816e41}} {{Gerrit|8e0b9ca}}, {{Gerrit|8dbc099}} ([[phab:T273226|T273226]])

=== 2021-10-27 ===
* 09:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6194d84}} ([[phab:T273226|T273226]])

=== 2021-10-25 ===
* 21:41 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|7e3343a}} ([[phab:T278156|T278156]])
* 20:22 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|ef0f537}} ([[phab:T284684|T284684]])
* 20:15 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|977236b}} ([[phab:T284183|T284183]])
* 18:47 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5dfef39}}, {{Gerrit|e940ae7}}, {{Gerrit|765b314}}, {{Gerrit|3e4c2de}}, {{Gerrit|c859c45}}, {{Gerrit|5526cb3}}

=== 2021-06-11 ===
* 15:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|a753cb2}}
* 15:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5187a4d}}

=== 2021-05-31 ===
* 09:20 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|79e458d}}

=== 2021-05-22 ===
* 20:58 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|258aae9a}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|67c42f43}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d170ea1}}
* 20:24 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2f39621}}
* 20:21 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b52079f}}
* 17:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|8ffbad82c3}} ([[phab:T279236|T279236]])
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6de2308}} and {{Gerrit|c436760}} ([[phab:T279236|T279236]])
* 16:06 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0966be04}} ([[phab:T279236|T279236]])
* 15:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9727a01c}}
* 14:02 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|69048ef}}
* 13:00 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|633eeb0f}}
* 12:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d4fa363}} ([[phab:T283287|T283287]])

=== 2021-04-03 ===
* 21:34 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|666222b}} and {{Gerrit|0dcafff}} ([[phab:T279236|T279236]])
* 21:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])
* 16:45 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])

=== 2021-03-26 ===
* 20:57 wm-bot: &lt;jeanfred&gt; Reinstalled requirements in both virtual environments for [[phab:T240312|T240312]]
* 20:55 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|bcc4fe41}} ([[phab:T240312|T240312]])

=== 2020-07-06 ===
* 22:55 wm-bot: &lt;root&gt; Migrated .webservicerc to service.template ([[phab:T257229|T257229]])

=== 2020-06-18 ===
* 11:01 wm-bot: &lt;jeanfred&gt; Restarted with --canonical for Toolforge domain migration

=== 2020-05-01 ===
* 22:59 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|f0db935}}, {{Gerrit|f8d9fdf}}, {{Gerrit|f57c060}}, {{Gerrit|6c534b7}} ([[phab:T248788|T248788]])

=== 2020-04-29 ===
* 20:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eabbac8}}, {{Gerrit|b74b352}} ([[phab:T248788|T248788]])

=== 2020-04-24 ===
* 22:22 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6963040}}, {{Gerrit|35a3ea7}}, {{Gerrit|22e27b5}}, {{Gerrit|9dd44ff}}, {{Gerrit|0e45815}}, {{Gerrit|6f36ffd}} ([[phab:T248788|T248788]])
* 19:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|9fea19b}}, {{Gerrit|f38047e}}

=== 2020-02-25 ===
* 23:35 wm-bot: &lt;root&gt; Migrated to 2020 Kubernetes cluster

=== 2020-02-16 ===
* 16:49 JeanFred: &quot;Deploy latest from Git master: {{Gerrit|0fcb522}} ([[phab:T243780|T243780]]), {{Gerrit|4cb6a5a}} ([[phab:T243780|T243780]], [[phab:T239067|T239067]])&quot;

=== 2020-02-11 ===
* 21:23 wm-bot157: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|c350e4f}} ([[phab:T243998|T243998]]), {{Gerrit|fa12c21}}

=== 2020-02-05 ===
* 12:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8977958}} ([[phab:T243780|T243780]])

=== 2020-02-03 ===
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|3b2867f}}, {{Gerrit|10f67fd}}, {{Gerrit|2e6a669}} ([[phab:T244030|T244030]]), {{Gerrit|279c92b}}, {{Gerrit|de6308e}}

=== 2020-01-16 ===
* 21:44 wm-bot: &lt;jeanfred&gt; Perform webservice restart for [[phab:T242967|T242967]]

=== 2019-12-11 ===
* 15:15 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|257618f1}} ([[phab:T240312|T240312]])

=== 2019-12-02 ===
* 06:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|770cc93}} ([[phab:T237182|T237182]])

=== 2019-11-25 ===
* 11:36 wm-bot: &lt;jeanfred&gt; Trigger a manual update of all dashboards ([[phab:T239085|T239085]])
* 11:34 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|684fb77}} ([[phab:T239085|T239085]])

=== 2019-11-21 ===
* 08:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2f94256}} ([[phab:T237187|T237187]])

=== 2019-11-10 ===
* 15:31 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|ab714de}} ([[phab:T224226|T224226]])

=== 2019-11-06 ===
* 09:12 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|7202182}} ([[phab:T237187|T237187]])
* 09:01 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|29b19c3}} ([[phab:T237182|T237182]])

=== 2019-11-04 ===
* 17:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d0f4222}} ([[phab:T237189|T237189]])
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|588dae8}}
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|471deddd}} ([[phab:T237182|T237182]])

=== 2019-10-31 ===
* 09:06 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|46484f5}} ([[phab:T224226|T224226]])
* 09:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2d71154}}, {{Gerrit|d0d2937}}, {{Gerrit|905007b}}, {{Gerrit|effee36}}, {{Gerrit|7c15083}}

=== 2019-10-30 ===
* 19:42 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|289a41b}}, {{Gerrit|02a1a4f}}, {{Gerrit|e8d4363}} ([[phab:T224226|T224226]])

=== 2019-10-25 ===
* 15:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|82de2957}} ([[phab:T224212|T224212]], [[phab:T223930|T223930]])
* 14:08 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6a055c2}} ([[phab:T228405|T228405]])
* 13:24 wm-bot: &lt;jeanfred&gt; Deploying from master, will break things.

=== 2019-09-19 ===
* 18:38 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d214f09}}, {{Gerrit|9b8dfa5}}
* 17:27 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|12ebc39}}

=== 2019-07-07 ===
* 22:49 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|5b4a1c6}}, {{Gerrit|ab90227}}, {{Gerrit|d101ca1}}, {{Gerrit|394be6a}}, {{Gerrit|ae16d68}}, {{Gerrit|728a8d2}}

=== 2019-05-30 ===
* 12:08 wm-bot: &lt;jeanfred&gt; Nuked the virtualenv and reinstalled all deps from scratch, in desperation for [[phab:T224651|T224651]]
* 10:59 wm-bot: &lt;jeanfred&gt; Service stop, mv logs, service start for [[phab:T224651|T224651]]

=== 2019-05-22 ===
* 16:39 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8e76a8a}}
* 13:13 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2fdede2}}, {{Gerrit|6b6c07c}}, {{Gerrit|c17cb78}} ([[phab:T224090|T224090]])

=== 2019-05-20 ===
* 19:01 wm-bot: &lt;jeanfred&gt; Restart uwsgi with 10 workers rather than the default of 4
* 19:00 wm-bot: &lt;jeanfred&gt; --help


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>r0lvux45gut6xxew10adak7ho0c6hzd</sha1>
    </revision>
    <revision>
      <id>2408836</id>
      <parentid>2408835</parentid>
      <timestamp>2026-05-03T19:44:52Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>wmbot~jeanfred@tools-bastion-15: Deplot 0fd3dd4 (Prepend default label column in Listeria output) for T368001</comment>
      <origin>2408836</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="24560" sha1="henfdy0c3enwe6ym5mtuzg2h19qz2co" xml:space="preserve">=== 2026-05-03 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deplot {{Gerrit|0fd3dd4}} (Prepend default label column in Listeria output) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|18813aa}} (Put label and description columns first in Listeria output) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|b41db93}} (Re-resolve grouping links after year rebinning) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fbf0bd}} (Add grouping_link_mode config and wire up page creation) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|729decc}} (Add GroupingPageCreator to create Listeria subpages) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|74bf18f}} (Add Listeria wikitext formatting to Grouping rows) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9b32a82}} (Split retrieve_and_process_data in process_page) for [[phab:T368001|T368001]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4dcc1a0}} (Extract prepare_report_groupings from process_data in PropertyStatistics) for [[phab:T368001|T368001]]
* 19:41 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|928f398}} (Remove obsolete version attribute from docker-compose.yml)

=== 2026-05-01 ===
* 18:16 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|01e97d6}} (Extract grouping link strategies into GroupingLink hierarchy), {{Gerrit|359770d}} (Remove pass-through methods for grouping link in GroupingConfiguration), {{Gerrit|54e4a7b}} (Add lang parameter to get_label_for_variable), {{Gerrit|cccd661}} (Simplify SPARQL for mul language in get_label_for_variable), {{Gerrit|548383d}} (Support any label language for grouping links), {{Gerrit|d571596}} (Support QID-based grouping links), {{Gerrit|cfd7c68}} (
* 18:15 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|19b856d}}, {{Gerrit|473fbeb}}, {{Gerrit|0cb0915}}

=== 2026-03-27 ===
* 22:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2bad8b3}} (Catch ModuleNotFoundError when unpickling stale cache entries)

=== 2026-03-24 ===
* 16:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|7883b73}} (Switch from bare imports to relative imports)
* 15:56 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5dbe412}} (Resolve grouping type eagerly in PropertyStatistics constructor)

=== 2026-03-23 ===
* 22:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|1b60d39}} (Handle stale pickle cache entries referencing removed classes)
* 22:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|124f4ca}} (Replace wikibase:isSomeValue with STRSTARTS filter for QLever compatibility)
* 21:28 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fd5828f}} (Detect grouping type via SPARQL and flatten to single GroupingConfiguration)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|e9fae82}} (Add AbstractGroupingType base class for grouping type strategies)
* 21:27 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|fee0ed5}} (Extract GroupingType strategy classes from GroupingConfiguration hierarchy)

=== 2026-03-18 ===
* 20:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f9dfb79}} (Add /healthz endpoint for automatic health check)

=== 2026-03-17 ===
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2940962}} (Replace GROUP_MAPPING Enum with SPECIAL_GROUPINGS tuple)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|420658b}} (Add MARKER attribute to special grouping classes)
* 16:51 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|976599f}} (Rename UnknownValueGrouping.TITLE to MARKER)

=== 2026-03-14 ===
* 16:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be09048}} (Add Wikimedia Commons support using QLever endpoint) for [[phab:T294893|T294893]]
* 16:38 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|ae53e61}} (Derive QLever UI URL from engine endpoint)
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5134f77}} (Use ka/Ma unit suffixes for geological-scale year grouping headings) for [[phab:T236590|T236590]]
* 15:45 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8a735a0}} (Add automatic date resolution rebinning for year groupings) for [[phab:T236590|T236590]]
* 15:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|6478051}}, {{Gerrit|4206c09}}, {{Gerrit|0e920fe}}, {{Gerrit|4eefea1}}

=== 2026-03-13 ===
* 20:33 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a39f1d9}} (Add proper handling for transient Wikidata server errors) for [[phab:T415439|T415439]]
* 20:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|0c418a5}} (Fix tests hitting live Wikidata API (and potential rate-limits)) for [[phab:T420054|T420054]]

=== 2026-03-11 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9665d8f}} (Remove obsolete GroupingType enum from column.py)
* 21:57 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|60d500a}} (Remove backward-compatible formatting wrapper methods from PropertyStatistics)
* 21:20 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|a452c28}} (Extract formatting logic into separate ResultsFormatter class)
* 17:29 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|02338eb}} (Move heading strings to class constants in Grouping objects) and {{Gerrit|d730307}} (Format NoGroup and Totals in  with the other groupings)

=== 2026-03-09 ===
* 18:32 wmbot~jeanfred@tools-bastion-15: Recreated Python 3.11 virtual environment using 'toolforge webservice python3.11 shell' / 'webservice-python-bootstrap --fresh' / '/data/project/integraality/www/python/venv/bin/pip install -r /data/project/integraality/integraality/requirements.txt'
* 18:30 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|55f1132}} (Switch to Python 3.11 as default interpreter)

=== 2026-03-08 ===
* 11:52 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|37fcc25}} (Extract jobs-framework runtime image as its own variable) and {{Gerrit|1f9bfbb}} (Add service.template deployment to Ansible)

=== 2026-03-06 ===
* 10:08 wmbot~jeanfred@tools-bastion-15: Successfully installed flask-3.1.3 mwparserfromhell-0.7.2 pywikibot-10.7.6 redis-7.0.1 (using a webservice python3.9 shell)
* 10:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9fa536f}}, {{Gerrit|9e5111c}}, {{Gerrit|e176f18}}, {{Gerrit|eff3045}}, {{Gerrit|8532e32}} (Dependencies upgrades)

=== 2026-03-05 ===
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5075934}} (Add support for user-specified groupings via template parameter) for [[phab:T419059|T419059]]
* 19:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|3c9fa98}} (Remove two debug print statements)
* 18:44 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28d800e}} (Extract rdfs:label SPARQL fragment from grouping.py to get_label_for_variable utility), {{Gerrit|40c39de}} (Make intermediate label variables unique to avoid clashes in get_label_for_variable) and {{Gerrit|64e8f94}} (Replace SERVICE wikibase:label with native SPARQL for QLever compatibility) for [[phab:T385749|T385749]]

=== 2025-11-03 ===
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]
* 17:19 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|2d3e128}} (Display the name of the SPARQL endpoint used in the edit summary) for [[phab:T385749|T385749]]

=== 2025-10-29 ===
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f40974f}} (Add links to QLever in the Queries) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|28c2e1b}} (Add link to QLever on queries error page) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4cc2bcc}} (Add configurable SPARQL endpoint support for QLever in PagesProcessor) for [[phab:T385749|T385749]]
* 16:49 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|be5c234}} (Add support for QLever as alternative SPARQL query engine) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|f34624d}} (Move pywikibot exception handling to SPARQL engine class) for [[phab:T385749|T385749]]
* 12:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|4da9f33}} (Implement SPARQL query engine dependency injection) for [[phab:T385749|T385749]]

=== 2025-10-28 ===
* 21:58 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|92ab962}} (Update pywikibot from 10.0.0 to latest 10.6.0)
* 21:34 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|77deb17}} (Remove link to 'SGE-jobs' tool in footer)

=== 2025-10-27 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8db01c6}} (Simplify Flask webserver startup in app.py)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|31f14b0}} (Bump ruff pre-commit hook from 0.9.3 to latest 0.14.2)
* 20:32 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|995121d}} (Bump uv pre-commit hook from 0.5.24 to latest 0.9.5 and regenerate requirements files)
* 09:07 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|69669c0}} (Optimize 'get_grouping_information_query' query using subquery) for [[phab:T400480|T400480]]

=== 2025-10-25 ===
* 09:00 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|9ac4c1f}} (Convert SPARQL query tests to use triple-quoted strings in PropertyStatisticsTest)

=== 2025-10-24 ===
* 21:02 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|5511fd0}} (Fix test_get_grouping_information_query_with_grouping_link for YearGrouping)
* 19:39 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|8bb5d37}} (Convert SPARQL query tests to use triple-quoted strings)
* 19:37 wmbot~jeanfred@tools-bastion-15: Deploy {{Gerrit|792b38e}} (Remove redundant GROUP BY parameter in 'get_grouping_information_query')

=== 2025-05-06 ===
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|284a70a}} (Update pywikibot from 9.6.1 to latest 10.0.0)
* 20:05 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|1f030a9}} (Re-generate uv lockfile to add 'upload-time' markers)
* 11:19 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8f2391e}} (Replace deprecated  with  in two unit tests)

=== 2025-05-04 ===
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7a2fa93}} (Fallback to mul when formatting grouping links) for [[phab:T393166|T393166]]
* 12:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)

=== 2025-05-03 ===
* 19:57 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|781934d}} (Move grouping-link to Grouping objects and retrieve via SPARQL) for [[phab:T237276|T237276]]

=== 2025-05-02 ===
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|381d8f9}} (Add pre-commit hook to run Python linter ruff)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5bc6fe9}} (Remove unused local variable in GroupingTest.test unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|f5f0044}} (Remove unused local variable in  unit-test)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a742a2}} (Remove unused column_key local variable from get_query_for_items_for_property_[posi{{!}}nega]tive)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|620ac00}} (Remove unused patcher in QueriesTests setUp method)
* 19:41 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|e32b9b9}} (Allow to optionally pass wikiproject data to SitelinkColumn constructor)

=== 2025-05-01 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|2a91226}} (Catch WDQS Timeouts and raise proper exception for all SPARQL queries) for [[phab:T278156|T278156]]
* 20:29 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|397c5e8}} (Also catch ServerError to handle WDQS timeouts) for [[phab:T384882|T384882]]

=== 2025-04-28 ===
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|eba498e}} (Use correct namespace when using pywikibot exceptions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ea55660}} (Add unit-tests for page_saving.py functions)
* 19:28 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|58918e6}} (Tweak linebreak in get_grouping_information_query which was breaking unit tests)

=== 2025-01-23 ===
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|a76ceec}} (Switch from Pipenv to uv as project manager)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5941f3f}} (Upgrade all Python dev-dependencies)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix 'QueriesTests' unit tests in test_app)
* 20:43 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|9268348}} (Fix  unit tests in test_app)

=== 2025-01-17 ===
* 10:03 wmbot~jeanfred@tools-sgebastion-10: Recreated virtual environment using 'toolforge webservice python3.9 shell' / 'webservice-python-bootstrap --fresh'

=== 2025-01-16 ===
* 16:36 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|525c9ed}} (Upgrade all Python dependencies)

=== 2024-07-01 ===
* 19:54 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8eecfc6}} &amp; {{Gerrit|0466207}}

=== 2024-06-06 ===
* 12:47 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|022fa2f}} ([[phab:T312727|T312727]])
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|ec28a13}}
* 12:46 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|7adc07f}}

=== 2024-05-10 ===
* 15:57 wmbot~multichill@tools-bastion-12: Leaving project

=== 2024-05-04 ===
* 20:22 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|51ba697}}
* 20:07 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|8ec2fec}} ([[phab:T251008|T251008]])

=== 2024-05-03 ===
* 12:01 wmbot~multichill@tools-bastion-12: emails: all -

=== 2024-04-30 ===
* 19:10 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|5a91a78}}, {{Gerrit|864d518}}, c9b199 ([[phab:T251008|T251008]]), {{Gerrit|0dd731f}}

=== 2024-03-17 ===
* 21:02 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|3a93a59}} ([[phab:T319813|T319813]])
* 20:25 wmbot~jeanfred@tools-sgebastion-10: Re-created virtual environment for Python 3.9 ([[phab:T319813|T319813]])
* 19:26 wmbot~jeanfred@tools-sgebastion-10: Deploy {{Gerrit|bde5715}}, {{Gerrit|9bdfb02}} ([[phab:T319813|T319813]])

=== 2023-11-21 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|992670c}} ([[phab:T351574|T351574]])
* 19:55 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|1f41a79}}, {{Gerrit|38e7623}}

=== 2023-10-17 ===
* 14:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d7f09b3}}, {{Gerrit|eb1de62}}, {{Gerrit|f4e3656}}, {{Gerrit|82f90c4}} ([[phab:T312726|T312726]])

=== 2023-10-03 ===
* 21:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 19:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3ba5e84}} ([[phab:T312729|T312729]])
* 18:04 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|da99820}}, {{Gerrit|801ddeb}}, {{Gerrit|f2ffd22}} ([[phab:T312729|T312729]])

=== 2023-10-02 ===
* 09:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3f9a3d5}}

=== 2023-09-26 ===
* 18:25 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3caa9ed}}, {{Gerrit|a8029ed}}, {{Gerrit|c88ad35}}, {{Gerrit|e575508}}, {{Gerrit|3e3f47f}}, {{Gerrit|63ab297}}, {{Gerrit|06cb52d}}, {{Gerrit|ac64840}}, {{Gerrit|6178698}} ([[phab:T312728|T312728]])

=== 2023-06-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5aecae8}}, {{Gerrit|649259f}}, {{Gerrit|9712666}} ([[phab:T338684|T338684]])

=== 2022-12-26 ===
* 10:48 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c90f9ef}} ([[phab:T325936|T325936]])

=== 2022-09-11 ===
* 12:50 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|bcbd306}}

=== 2022-07-12 ===
* 15:04 wm-bot: &lt;jeanfred&gt; Triggering full update
* 15:02 wm-bot: &lt;jeanfred&gt; jstop {{Gerrit|5976623}} to kill weekly update job stuck since May 22

=== 2022-06-03 ===
* 12:37 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9e85ede}} ([[phab:T309861|T309861]])

=== 2022-05-15 ===
* 21:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|3a675b6}}
* 20:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|80f073e}}
* 19:31 wm-bot: &lt;jeanfred&gt; Uninstall removed requirements and restart webservice
* 19:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|f6dc34b}}
* 19:15 wm-bot: &lt;jeanfred&gt; Install all requirements and restart webservice
* 19:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d395b4c}}, {{Gerrit|591c450}}, {{Gerrit|6d09e5b}}

=== 2022-05-14 ===
* 14:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2a521d1}}, {{Gerrit|29d6b62}}, {{Gerrit|8bd793d}} ([[phab:T257942|T257942]])

=== 2022-05-11 ===
* 19:56 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|43f07d2}}, {{Gerrit|da1f2ef}}, {{Gerrit|6ab7547}}, {{Gerrit|5f6b20a}}, {{Gerrit|0df2121}}

=== 2022-01-28 ===
* 20:44 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|13ed7a3}}

=== 2021-11-01 ===
* 20:30 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|972a1df}}

=== 2021-10-30 ===
* 19:36 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|c1628b9}} and {{Gerrit|b6940b0}}

=== 2021-10-29 ===
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|40c8cd3}}
* 09:49 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|af49bab}} ([[phab:T236590|T236590]] and [[phab:T273226|T273226]])
* 09:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2260c8f}} (follow up to [[phab:T294570|T294570]])
* 09:16 wm-bot: &lt;jeanfred&gt; Install requirements in py37 venv ([[phab:T294570|T294570]])
* 09:14 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a4560a}} and {{Gerrit|de296cd}} ([[phab:T294570|T294570]])

=== 2021-10-28 ===
* 17:14 wm-bot: &lt;jeanfred&gt; Rollbacked to {{Gerrit|977236b42daef66818e561b49e64264a0c785454}} for [[phab:T294570|T294570]] and restart
* 12:38 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0a8e7fb}} ([[phab:T236590|T236590]])
* 11:31 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|30a2f2ba}} ([[phab:T278156|T278156]])
* 11:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5c547d8}}
* 10:18 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b4f12bc}} to help investigate [[phab:T236960|T236960]]
* 09:08 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|137f5b8}}, {{Gerrit|c816e41}} {{Gerrit|8e0b9ca}}, {{Gerrit|8dbc099}} ([[phab:T273226|T273226]])

=== 2021-10-27 ===
* 09:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6194d84}} ([[phab:T273226|T273226]])

=== 2021-10-25 ===
* 21:41 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|7e3343a}} ([[phab:T278156|T278156]])
* 20:22 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|ef0f537}} ([[phab:T284684|T284684]])
* 20:15 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|977236b}} ([[phab:T284183|T284183]])
* 18:47 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5dfef39}}, {{Gerrit|e940ae7}}, {{Gerrit|765b314}}, {{Gerrit|3e4c2de}}, {{Gerrit|c859c45}}, {{Gerrit|5526cb3}}

=== 2021-06-11 ===
* 15:32 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|a753cb2}}
* 15:29 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|5187a4d}}

=== 2021-05-31 ===
* 09:20 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|79e458d}}

=== 2021-05-22 ===
* 20:58 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|258aae9a}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|67c42f43}}
* 20:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d170ea1}}
* 20:24 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|2f39621}}
* 20:21 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|b52079f}}
* 17:28 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|8ffbad82c3}} ([[phab:T279236|T279236]])
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|6de2308}} and {{Gerrit|c436760}} ([[phab:T279236|T279236]])
* 16:06 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|0966be04}} ([[phab:T279236|T279236]])
* 15:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|9727a01c}}
* 14:02 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|69048ef}}
* 13:00 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|633eeb0f}}
* 12:59 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|d4fa363}} ([[phab:T283287|T283287]])

=== 2021-04-03 ===
* 21:34 wm-bot: &lt;jeanfred&gt; Deploy {{Gerrit|666222b}} and {{Gerrit|0dcafff}} ([[phab:T279236|T279236]])
* 21:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])
* 16:45 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eac4d2c}} ([[phab:T257942|T257942]])

=== 2021-03-26 ===
* 20:57 wm-bot: &lt;jeanfred&gt; Reinstalled requirements in both virtual environments for [[phab:T240312|T240312]]
* 20:55 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|bcc4fe41}} ([[phab:T240312|T240312]])

=== 2020-07-06 ===
* 22:55 wm-bot: &lt;root&gt; Migrated .webservicerc to service.template ([[phab:T257229|T257229]])

=== 2020-06-18 ===
* 11:01 wm-bot: &lt;jeanfred&gt; Restarted with --canonical for Toolforge domain migration

=== 2020-05-01 ===
* 22:59 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|f0db935}}, {{Gerrit|f8d9fdf}}, {{Gerrit|f57c060}}, {{Gerrit|6c534b7}} ([[phab:T248788|T248788]])

=== 2020-04-29 ===
* 20:33 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|eabbac8}}, {{Gerrit|b74b352}} ([[phab:T248788|T248788]])

=== 2020-04-24 ===
* 22:22 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6963040}}, {{Gerrit|35a3ea7}}, {{Gerrit|22e27b5}}, {{Gerrit|9dd44ff}}, {{Gerrit|0e45815}}, {{Gerrit|6f36ffd}} ([[phab:T248788|T248788]])
* 19:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|9fea19b}}, {{Gerrit|f38047e}}

=== 2020-02-25 ===
* 23:35 wm-bot: &lt;root&gt; Migrated to 2020 Kubernetes cluster

=== 2020-02-16 ===
* 16:49 JeanFred: &quot;Deploy latest from Git master: {{Gerrit|0fcb522}} ([[phab:T243780|T243780]]), {{Gerrit|4cb6a5a}} ([[phab:T243780|T243780]], [[phab:T239067|T239067]])&quot;

=== 2020-02-11 ===
* 21:23 wm-bot157: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|c350e4f}} ([[phab:T243998|T243998]]), {{Gerrit|fa12c21}}

=== 2020-02-05 ===
* 12:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8977958}} ([[phab:T243780|T243780]])

=== 2020-02-03 ===
* 16:57 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|3b2867f}}, {{Gerrit|10f67fd}}, {{Gerrit|2e6a669}} ([[phab:T244030|T244030]]), {{Gerrit|279c92b}}, {{Gerrit|de6308e}}

=== 2020-01-16 ===
* 21:44 wm-bot: &lt;jeanfred&gt; Perform webservice restart for [[phab:T242967|T242967]]

=== 2019-12-11 ===
* 15:15 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|257618f1}} ([[phab:T240312|T240312]])

=== 2019-12-02 ===
* 06:40 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|770cc93}} ([[phab:T237182|T237182]])

=== 2019-11-25 ===
* 11:36 wm-bot: &lt;jeanfred&gt; Trigger a manual update of all dashboards ([[phab:T239085|T239085]])
* 11:34 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|684fb77}} ([[phab:T239085|T239085]])

=== 2019-11-21 ===
* 08:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2f94256}} ([[phab:T237187|T237187]])

=== 2019-11-10 ===
* 15:31 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|ab714de}} ([[phab:T224226|T224226]])

=== 2019-11-06 ===
* 09:12 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|7202182}} ([[phab:T237187|T237187]])
* 09:01 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|29b19c3}} ([[phab:T237182|T237182]])

=== 2019-11-04 ===
* 17:46 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d0f4222}} ([[phab:T237189|T237189]])
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|588dae8}}
* 17:35 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|471deddd}} ([[phab:T237182|T237182]])

=== 2019-10-31 ===
* 09:06 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|46484f5}} ([[phab:T224226|T224226]])
* 09:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2d71154}}, {{Gerrit|d0d2937}}, {{Gerrit|905007b}}, {{Gerrit|effee36}}, {{Gerrit|7c15083}}

=== 2019-10-30 ===
* 19:42 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|289a41b}}, {{Gerrit|02a1a4f}}, {{Gerrit|e8d4363}} ([[phab:T224226|T224226]])

=== 2019-10-25 ===
* 15:02 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|82de2957}} ([[phab:T224212|T224212]], [[phab:T223930|T223930]])
* 14:08 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|6a055c2}} ([[phab:T228405|T228405]])
* 13:24 wm-bot: &lt;jeanfred&gt; Deploying from master, will break things.

=== 2019-09-19 ===
* 18:38 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|d214f09}}, {{Gerrit|9b8dfa5}}
* 17:27 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|12ebc39}}

=== 2019-07-07 ===
* 22:49 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|5b4a1c6}}, {{Gerrit|ab90227}}, {{Gerrit|d101ca1}}, {{Gerrit|394be6a}}, {{Gerrit|ae16d68}}, {{Gerrit|728a8d2}}

=== 2019-05-30 ===
* 12:08 wm-bot: &lt;jeanfred&gt; Nuked the virtualenv and reinstalled all deps from scratch, in desperation for [[phab:T224651|T224651]]
* 10:59 wm-bot: &lt;jeanfred&gt; Service stop, mv logs, service start for [[phab:T224651|T224651]]

=== 2019-05-22 ===
* 16:39 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|8e76a8a}}
* 13:13 wm-bot: &lt;jeanfred&gt; Deploy latest from Git master: {{Gerrit|2fdede2}}, {{Gerrit|6b6c07c}}, {{Gerrit|c17cb78}} ([[phab:T224090|T224090]])

=== 2019-05-20 ===
* 19:01 wm-bot: &lt;jeanfred&gt; Restart uwsgi with 10 workers rather than the default of 4
* 19:00 wm-bot: &lt;jeanfred&gt; --help


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>henfdy0c3enwe6ym5mtuzg2h19qz2co</sha1>
    </revision>
  </page>
  <page>
    <title>Talk:Annual Toolforge Survey</title>
    <ns>1</ns>
    <id>447099</id>
    <revision>
      <id>2408840</id>
      <parentid>2252551</parentid>
      <timestamp>2026-05-04T05:19:11Z</timestamp>
      <contributor>
        <username>SSapaty (WMF)</username>
        <id>28955</id>
      </contributor>
      <comment>/* What user name to opt out with? */ Reply</comment>
      <origin>2408840</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="2362" sha1="ik7zn6kvhopz9b0m9u070t3kzlv1j97" xml:space="preserve">== No Google Form please ==

Dear {{Ping|Jhernandez}} ([[m:User:JHernandez (WMF)]]),

Thank you for your email. Apologies but I do not want to fill the Google Form module I received via email from your email address for various reasons.

Some of them:

* https://www.gnu.org/proprietary/malware-google.en.html
* https://www.gnu.org/philosophy/javascript-trap.html

Please understand my perspective and allow me to somehow participate in this survey without executing any proprietary software on my computer.

(Unfortunately I have been reporting the same issue for years...)

Having said that, I am available to help you to adopt whatever freedom solution, such as [[w:LimeSurvey]].

Warm wishes --[[User:Valerio Bozzolan|Valerio Bozzolan]] ([[User talk:Valerio Bozzolan|talk]]) 09:12, 1 February 2021 (UTC)
:Hi Valerio, I completely understand your perspective and would like to support you. I'll reach out by email directly. Thanks for your input --[[User:Jhernandez|Jhernandez]] ([[User talk:Jhernandez|talk]]) 11:15, 1 February 2021 (UTC)
:: Thank you again, you were very kind with me :) I'm trying to prevent this in the future in order to save mine and your precious time. Thank you again for your help. I will ask for help in [[m:Wikimedia Forum]] --[[User:Valerio Bozzolan|Valerio Bozzolan]] ([[User talk:Valerio Bozzolan|talk]]) 10:49, 15 February 2021 (UTC)

== What user name to opt out with? ==

The email I received tells me that I have to add my &lt;q&gt;Wikimedia developer account name (i.e. the username you use to log into wikitech.wikimedia.org)&lt;/q&gt; to the list at [[Annual Toolforge Survey/Opt out]]. As you can see from the site notice, [[Wikitech/SUL-migration|the two are no longer the same]]: Wikitech has been separated from LDAP (developer account) and is being migrated to SUL. So which part of the sentence is true, &lt;q&gt;Wikimedia developer account name&lt;/q&gt; or &lt;q&gt;the username you use to log into wikitech.wikimedia.org&lt;/q&gt;? (Note that I don’t want to opt out, and my user names are the same in both systems anyway, but others could be confused.) —[[User:Tacsipacsi|Tacsipacsi]] ([[User talk:Tacsipacsi|talk]]) 16:10, 6 December 2024 (UTC)

:Sorry for the late response.
:The language has been clarified to say &quot;your LDAP username&quot;.
:Thanks! [[User:SSapaty (WMF)|SSapaty (WMF)]] ([[User talk:SSapaty (WMF)|talk]]) 05:19, 4 May 2026 (UTC)</text>
      <sha1>ik7zn6kvhopz9b0m9u070t3kzlv1j97</sha1>
    </revision>
  </page>
  <page>
    <title>Map of database maintenance</title>
    <ns>0</ns>
    <id>449160</id>
    <revision>
      <id>2408837</id>
      <parentid>2408753</parentid>
      <timestamp>2026-05-04T00:02:39Z</timestamp>
      <contributor>
        <username>Dexbot</username>
        <id>30554</id>
      </contributor>
      <minor />
      <comment>Bot: Updating the report</comment>
      <origin>2408837</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="983" sha1="ki7h1jj8rvlklo6l64v35b8zvw07skk" xml:space="preserve">{{/Header}}
== Today (2026-05-04) ==
== Yesterday (2026-05-03) ==
== Last seven days ==
{| class=&quot;wikitable&quot;
|+ eqiad
|-
! Section !! Work
|-
| s1 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
| s5 || 
* [[phab:T419635|Drop il_to column from imagelinks table in wmf production (T419635)]] (fceratto)
* [[phab:T424550|Switchover s5 master (db1230 -&amp;gt; db1210) (T424550)]] (marostegui)
|-
| s6 || [[phab:T424522|Switchover s6 master (db1173 -&amp;gt; db1201) (T424522)]] (marostegui)
|-
| s7 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
|}
{| class=&quot;wikitable&quot;
|+ codfw
|-
! Section !! Work
|-
| s1 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
| s5 || 
* [[phab:T419635|Drop il_to column from imagelinks table in wmf production (T419635)]] (fceratto)
* [[phab:T424521|Switchover s5 master (db2213 -&amp;gt; db2192) (T424521)]] (marostegui)
|-
| s7 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
| s8 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
|}


[[Category:MariaDB]]</text>
      <sha1>ki7h1jj8rvlklo6l64v35b8zvw07skk</sha1>
    </revision>
    <revision>
      <id>2408919</id>
      <parentid>2408837</parentid>
      <timestamp>2026-05-04T08:21:29Z</timestamp>
      <contributor>
        <username>Dexbot</username>
        <id>30554</id>
      </contributor>
      <minor />
      <comment>Bot: Updating the report</comment>
      <origin>2408919</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="1150" sha1="fvds15c9pyu41aipjmjk93wcgry11vc" xml:space="preserve">{{/Header}}
== Today (2026-05-04) ==
{| class=&quot;wikitable&quot;
|+ eqiad
|-
! Section !! Work
|-
| s8 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
|}
== Yesterday (2026-05-03) ==
== Last seven days ==
{| class=&quot;wikitable&quot;
|+ eqiad
|-
! Section !! Work
|-
| s1 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
| s5 || 
* [[phab:T419635|Drop il_to column from imagelinks table in wmf production (T419635)]] (fceratto)
* [[phab:T424550|Switchover s5 master (db1230 -&amp;gt; db1210) (T424550)]] (marostegui)
|-
| s6 || [[phab:T424522|Switchover s6 master (db1173 -&amp;gt; db1201) (T424522)]] (marostegui)
|-
| s7 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
| s8 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
|}
{| class=&quot;wikitable&quot;
|+ codfw
|-
! Section !! Work
|-
| s1 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
| s5 || 
* [[phab:T419635|Drop il_to column from imagelinks table in wmf production (T419635)]] (fceratto)
* [[phab:T424521|Switchover s5 master (db2213 -&amp;gt; db2192) (T424521)]] (marostegui)
|-
| s7 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
| s8 || [[phab:T419961|Login (T419961)]] (fceratto)
|-
|}


[[Category:MariaDB]]</text>
      <sha1>fvds15c9pyu41aipjmjk93wcgry11vc</sha1>
    </revision>
  </page>
  <page>
    <title>User talk:Ollie Shotton</title>
    <ns>3</ns>
    <id>457871</id>
    <redirect title="User talk:Ollie Hyde (WMDE)" />
    <revision>
      <id>2408869</id>
      <parentid>2271166</parentid>
      <timestamp>2026-05-04T07:20:48Z</timestamp>
      <contributor>
        <username>Dexbot</username>
        <id>30554</id>
      </contributor>
      <minor />
      <comment>Bot: Bot: Fixing double redirect to [[User talk:Ollie Hyde (WMDE)]]</comment>
      <origin>2408869</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="41" sha1="ily6klp13edhxq1i61eefwt8x04jzsv" xml:space="preserve">#REDIRECT [[User talk:Ollie Hyde (WMDE)]]</text>
      <sha1>ily6klp13edhxq1i61eefwt8x04jzsv</sha1>
    </revision>
  </page>
  <page>
    <title>Fundraising/SAL</title>
    <ns>0</ns>
    <id>459699</id>
    <revision>
      <id>2408818</id>
      <parentid>2408646</parentid>
      <timestamp>2026-05-03T13:15:18Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>ejegg|hackathon: civicrm upgraded from 0afbc8ea to 38dcf7a8</comment>
      <origin>2408818</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="22451" sha1="mkrblwl047g7lf0ypqas0ijk8yr9adp" xml:space="preserve">== 2026-05-03 ==
* hackathon: civicrm upgraded from {{Gerrit|0afbc8ea}} to {{Gerrit|38dcf7a8}}

== 2026-05-02 ==
* 09:44 eileen: civicrm upgraded from {{Gerrit|7556c5c7}} to {{Gerrit|0afbc8ea}}
* 09:43 eileen: SmashPig upgraded from {{Gerrit|88a1bcba}} to {{Gerrit|1be60746}}

== 2026-05-01 ==
* 16:36 eileen: civicrm upgraded from {{Gerrit|1a835879}} to {{Gerrit|7556c5c7}}
* 15:08 eileen: civicrm upgraded from {{Gerrit|9ed32632}} to {{Gerrit|1a835879}}
* 14:02 eileen: civicrm upgraded from {{Gerrit|081d5a29}} to {{Gerrit|9ed32632}}

== 2026-04-29 ==
* 13:51 jgleeson: payments-wiki upgraded from {{Gerrit|2e2eb8a2}} to {{Gerrit|4e0c944b}}
* 13:49 jgleeson: tools upgraded from {{Gerrit|f52a5dcf}} to {{Gerrit|afbd0f67}}

== 2026-04-28 ==
* 20:58 larssandergreen: civicrm upgraded from {{Gerrit|be3bb76b}} to {{Gerrit|081d5a29}}

== 2026-04-27 ==
* 19:04 ejegg: fundraising civicrm upgraded from {{Gerrit|3f8d49fa}} to {{Gerrit|be3bb76b}}
* 19:02 ejegg: payments-wiki upgraded from {{Gerrit|b1a352af}} to {{Gerrit|5265089d}}
* 18:58 ejegg: standalone (IPN listener) SmashPig upgraded from {{Gerrit|572b69da}} to {{Gerrit|88a1bcba}}

== 2026-04-23 ==
* 20:40 ejegg: payments-wiki upgraded from {{Gerrit|6e78ef91}} to {{Gerrit|b1a352af}}
* 19:52 ejegg: civicrm fundraising upgraded from {{Gerrit|5ea4c8d3}} to {{Gerrit|3f8d49fa}}
* 19:27 ejegg: SmashPig upgraded from {{Gerrit|f1b3f3d9}} to {{Gerrit|572b69da}}
* 16:32 larssandergreen: civicrm upgraded from {{Gerrit|53a0b46f}} to {{Gerrit|5ea4c8d3}}
* 16:31 larssandergreen: tools upgraded from {{Gerrit|edca3f63}} to {{Gerrit|f52a5dcf}}

== 2026-04-22 ==
* 02:10 eileen: civicrm upgraded from {{Gerrit|abd23ad7}} to {{Gerrit|53a0b46f}}

== 2026-04-21 ==
* 23:16 cstone: civicrm upgraded from {{Gerrit|22f24ae4}} to {{Gerrit|abd23ad7}}
* 19:46 larssandergreen: civicrm upgraded from {{Gerrit|ddc1f044}} to {{Gerrit|22f24ae4}}

== 2026-04-20 ==
* 19:21 larssandergreen: tools upgraded from {{Gerrit|26ab0125}} to {{Gerrit|edca3f63}}
* 15:09 ejegg: payments-wiki upgraded from {{Gerrit|86a42498}} to {{Gerrit|6e78ef91}}

== 2026-04-17 ==
* 01:08 larssandergreen: civicrm upgraded from {{Gerrit|90c0ccd9}} to {{Gerrit|ddc1f044}}

== 2026-04-16 ==
* 16:39 larssandergreen: tools upgraded from {{Gerrit|f14a814e}} to {{Gerrit|26ab0125}}
* 14:20 larssandergreen: civicrm upgraded from {{Gerrit|801847a7}} to {{Gerrit|90c0ccd9}}
* 14:19 larssandergreen: tools upgraded from {{Gerrit|9bff5f07}} to {{Gerrit|f14a814e}}
* 02:39 ejegg: standalone (IPN listener) SmashPig upgraded from {{Gerrit|61fee241}} to {{Gerrit|f1b3f3d9}}

== 2026-04-15 ==
* 21:50 eileen: civicrm upgraded from {{Gerrit|6f33b6d0}} to {{Gerrit|801847a7}}
* 06:14 eileen: civicrm upgraded from {{Gerrit|a047bf92}} to {{Gerrit|6f33b6d0}}
* 01:20 eileen: SmashPig upgraded from {{Gerrit|100101fb}} to {{Gerrit|61fee241}}

== 2026-04-14 ==
* 23:58 eileen: civicrm upgraded from {{Gerrit|eb3d73e4}} to {{Gerrit|a047bf92}}
* 23:48 wfan: payments-wiki upgraded from {{Gerrit|26f5451a}} to {{Gerrit|c3b34f99}}
* 22:31 eileen: civicrm upgraded from {{Gerrit|2058927e}} to {{Gerrit|eb3d73e4}}
* 22:25 eileen: civicrm upgraded from {{Gerrit|2058927e}} to {{Gerrit|eb3d73e4}}
* 18:06 ejegg: fundraising civicrm upgraded from {{Gerrit|fccf9b3a}} to {{Gerrit|2058927e}}

== 2026-04-13 ==
* 18:20 ejegg: fundraising civicrm upgraded from {{Gerrit|fa20eb0a}} to {{Gerrit|fccf9b3a}}
* 17:04 ejegg: re-enabled recurring donation charge jobs
* 16:29 ejegg: fundraising civicrm upgraded from {{Gerrit|eb188fa2}} to {{Gerrit|fa20eb0a}}
* 16:27 ejegg: disabled recurring donation charge jobs for code / settings update
* 12:44 jgleeson: donorwiki upgraded from {{Gerrit|064a770e}} to {{Gerrit|26f5451a}}

== 2026-04-10 ==
* 15:35 jgleeson: payments-wiki upgraded from {{Gerrit|c017d7e7}} to {{Gerrit|dd45f867}}

== 2026-04-09 ==
* 23:31 wfan: payments-wiki upgraded from {{Gerrit|064a770e}} to {{Gerrit|c017d7e7}}
* 21:49 ejegg: fundraising civicrm upgraded from {{Gerrit|3d3c0a62}} to {{Gerrit|eb188fa2}}
* 19:00 larssandergreen: tools upgraded from {{Gerrit|986f7f83}} to {{Gerrit|9bff5f07}}
* 13:08 jgleeson: civicrm upgraded from {{Gerrit|d8d3871c}} to {{Gerrit|3d3c0a62}}
* 11:53 jgleeson: SmashPig upgraded from {{Gerrit|5c083891}} to {{Gerrit|100101fb}}
* 01:20 ejegg: fundraising civicrm upgraded from {{Gerrit|e60321bb}} to {{Gerrit|d8d3871c}}

== 2026-04-08 ==
* 17:44 ejegg: fundraising civicrm upgraded from {{Gerrit|4ee0b5e8}} to {{Gerrit|e60321bb}}
* 15:12 ejegg: payments-wiki upgraded from {{Gerrit|1ad85e6c}} to {{Gerrit|064a770e}}
* 01:19 ejegg: donorwiki upgraded from {{Gerrit|1ad85e6c}} to {{Gerrit|064a770e}}
* 00:26 dwisehaupt: cloning new frdb frdb1008 from frdb2005

== 2026-04-07 ==
* 18:34 wfan: civicrm upgraded from {{Gerrit|9104e70b}} to {{Gerrit|6f762e29}}

== 2026-04-06 ==
* 20:42 ejegg: re-enabled recurring donation charge job
* 20:33 wfan: donorwiki upgraded from {{Gerrit|c2d03117}} to {{Gerrit|1ad85e6c}}
* 20:32 wfan: payments-wiki upgraded from {{Gerrit|80cda166}} to {{Gerrit|1ad85e6c}}
* 16:53 ejegg: disabled recurring donations charge job while diagnosing gr4vy routing errors
* 16:03 ejegg: civicrm upgraded from {{Gerrit|4ee11209}} to {{Gerrit|9104e70b}}

== 2026-04-03 ==
* 00:04 wfan: civicrm upgraded from {{Gerrit|49f541cd}} to {{Gerrit|4ee11209}}

== 2026-04-02 ==
* 21:38 cstone: payments-wiki upgraded from {{Gerrit|86bec442}} to {{Gerrit|80cda166}}
* 05:24 eileen: civicrm upgraded from {{Gerrit|c512abc6}} to {{Gerrit|49f541cd}}
* 02:39 eileen: civicrm upgraded from {{Gerrit|bbed1291}} to {{Gerrit|c512abc6}}
* 02:16 eileen: SmashPig upgraded from {{Gerrit|9af71a7c}} to {{Gerrit|18ea746a}}

== 2026-04-01 ==
* 18:57 eileen: civicrm upgraded from {{Gerrit|a1bf4768}} to {{Gerrit|bbed1291}}
* 04:11 eileen: civicrm upgraded from {{Gerrit|11a2f9ab}} to {{Gerrit|a1bf4768}}
* 03:18 ejegg: payments-wiki upgraded from {{Gerrit|02bf54b0}} to {{Gerrit|86bec442}}

== 2026-03-31 ==
* 22:03 jgleeson: tools upgraded from {{Gerrit|9985e723}} to {{Gerrit|986f7f83}}
* 20:16 eileen: civicrm upgraded from {{Gerrit|c3cc3562}} to {{Gerrit|b468301c}}
* 18:40 jgleeson: tools upgraded from {{Gerrit|161049ac}} to {{Gerrit|9985e723}}
* 17:39 ejegg: Standalone (IPN listener) SmashPig upgraded from {{Gerrit|abf8682a}} to {{Gerrit|9af71a7c}}
* 16:38 jgleeson: tools upgraded from {{Gerrit|f605b570}} to {{Gerrit|161049ac}}
* 16:24 jgleeson: donorwiki updated from {{Gerrit|d79a98b5}} to {{Gerrit|c2d03117}}
* 02:46 eileen: civicrm upgraded from {{Gerrit|591bef29}} to {{Gerrit|c3cc3562}}
* 01:00 eileen: civicrm upgraded from {{Gerrit|cf871dd3}} to {{Gerrit|591bef29}}

== 2026-03-30 ==
* 23:19 eileen: civicrm upgraded from {{Gerrit|7d299b48}} to {{Gerrit|cf871dd3}}
* 21:09 eileen: civicrm upgraded from {{Gerrit|3724cc2d}} to {{Gerrit|7d299b48}}
* 20:40 eileen: civicrm upgraded from {{Gerrit|58426b1e}} to {{Gerrit|3724cc2d}}
* 14:51 ejegg: standalone (IPN listener) SmashPig upgraded from {{Gerrit|545f0b10}} to {{Gerrit|abf8682a}}

== 2026-03-28 ==
* 02:57 ejegg: payments-wiki upgraded from {{Gerrit|d79a98b5}} to {{Gerrit|b239a6b7}}
* 02:27 eileen: civicrm upgraded from {{Gerrit|7138d524}} to {{Gerrit|58426b1e}}

== 2026-03-27 ==
* 22:37 eileen: civicrm upgraded from {{Gerrit|c51e98cc}} to {{Gerrit|7138d524}}
* 17:50 dwisehaupt: Corrected value: on frdb1005 running the following in mysql to up the buffer pool to ~412G: set global innodb_buffer_pool_size = 431669379072;
* 17:34 dwisehaupt: on frdb1005 running the following in mysql to up the buffer pool to ~412G: set global innodb_buffer_pool_size = 421552128;

== 2026-03-26 ==
* 20:00 jgleeson: donorwiki upgraded from {{Gerrit|48cc2e9d}} to {{Gerrit|d79a98b5}}
* 19:59 jgleeson: payments-wiki upgraded from {{Gerrit|b387c6ba}} to {{Gerrit|d79a98b5}}
* 19:48 ejegg: fundraising civicrm upgraded from {{Gerrit|88b497b8}} to {{Gerrit|db102c77}}
* 19:25 larssandergreen: civicrm upgraded from {{Gerrit|26dab2f0}} to {{Gerrit|88b497b8}}
* 16:26 jgleeson: SmashPig upgraded from {{Gerrit|5d8a0330}} to {{Gerrit|545f0b10}}
* 02:13 eileen: civicrm upgraded from {{Gerrit|a752f9e8}} to {{Gerrit|26dab2f0}}
* 00:05 eileen: civicrm upgraded from {{Gerrit|277fe75e}} to {{Gerrit|a752f9e8}}

== 2026-03-25 ==
* 21:53 eileen: civicrm upgraded from {{Gerrit|97319295}} to {{Gerrit|277fe75e}}
* 19:58 eileen: civicrm upgraded from {{Gerrit|d3dfedb4}} to {{Gerrit|97319295}}

== 2026-03-24 ==
* 19:32 wfan: payments-wiki upgraded from {{Gerrit|fce3fab5}} to {{Gerrit|b387c6ba}}
* 04:24 eileen: civicrm upgraded from {{Gerrit|0aef661f}} to {{Gerrit|b2ed875e}}
* 03:38 eileen: config revision changed from {{Gerrit|ded0c289}} to {{Gerrit|16592428}} schedule stripe download
* 03:33 eileen: config revision changed from {{Gerrit|79e052e4}} to {{Gerrit|ded0c289}} temporarily disable adyen audit parse - let's fix those misplaced IDs
* 01:55 eileen: civicrm upgraded from {{Gerrit|b2c7f1d0}} to {{Gerrit|0aef661f}}
* 00:08 eileen: civicrm upgraded from {{Gerrit|80344f51}} to {{Gerrit|b2c7f1d0}}

== 2026-03-23 ==
* 21:30 eileen: config revision changed from {{Gerrit|8c5587f3}} to {{Gerrit|2dd50e7c}}
* 18:55 wfan: civicrm upgraded from {{Gerrit|675455b2}} to {{Gerrit|80344f51}}
* 17:27 larssandergreen: tools upgraded from {{Gerrit|e60f63b3}} to {{Gerrit|f605b570}}
* 15:59 ejegg: civicrm upgraded from {{Gerrit|a2d4b17c}} to {{Gerrit|675455b2}}
* 12:25 jgleeson: payments-wiki upgraded from {{Gerrit|48cc2e9d}} to {{Gerrit|91d9eee9}}

== 2026-03-20 ==
* 00:34 eileen: * civicrm upgraded from {{Gerrit|adc36173}} to {{Gerrit|a2d4b17c}}
* 00:31 cstone: payments-wiki upgraded from {{Gerrit|f3420a6f}} to {{Gerrit|48cc2e9d}}
* 00:27 eileen: config revision changed from {{Gerrit|a7486f6a}} to {{Gerrit|a1a426f3}}
* 00:27 eileen: SmashPig upgraded from {{Gerrit|78a8e70a}} to {{Gerrit|5d8a0330}}

== 2026-03-19 ==
* 14:08 damilare: civiproxy upgraded from {{Gerrit|6625c844}} to {{Gerrit|38ba8348}}

== 2026-03-17 ==
* 18:40 jgleeson: donorwiki upgraded from {{Gerrit|4c09db39}} to {{Gerrit|7d1666f9}}
* 06:20 eileen: civicrm upgraded from {{Gerrit|7fe14629}} to {{Gerrit|adc36173}}
* 05:06 eileen: civicrm upgraded from {{Gerrit|e622a222}} to {{Gerrit|7fe14629}}
* 03:48 eileen: civicrm upgraded from {{Gerrit|5360f9ad}} to {{Gerrit|e622a222}}
* 02:13 eileen: civicrm upgraded from {{Gerrit|3283e3ca}} to {{Gerrit|e73c6b50}}

== 2026-03-15 ==
* 23:56 eileen: civicrm upgraded from {{Gerrit|dce257f0}} to {{Gerrit|3283e3ca}}
* 19:43 eileen: civicrm upgraded from {{Gerrit|a1279ee4}} to {{Gerrit|dce257f0}}

== 2026-03-13 ==
* ish: payments-wiki upgraded from {{Gerrit|f40a1153}} to {{Gerrit|f3420a6f}}

== 2026-03-11 ==
* 21:20 larssandergreen: civicrm upgraded from {{Gerrit|c2c716ca}} to {{Gerrit|a1279ee4}}
* 19:15 eileen: civicrm upgraded from {{Gerrit|81baf495}} to {{Gerrit|c2c716ca}}
* 07:18 eileen: config revision changed from {{Gerrit|ed2295ab}} to {{Gerrit|a7486f6a}}
* 07:02 eileen: civicrm upgraded from {{Gerrit|f418297f}} to {{Gerrit|81baf495}}
* 02:28 eileen: civicrm upgraded from {{Gerrit|14e8200e}} to {{Gerrit|da26f37d}}
* 00:06 eileen: civicrm upgraded from {{Gerrit|fbb38eda}} to {{Gerrit|14e8200e}}

== 2026-03-10 ==
* 22:08 eileen: civicrm upgraded from {{Gerrit|ef319ea3}} to {{Gerrit|fbb38eda}}
* 19:35 eileen: civicrm upgraded from {{Gerrit|773d9fb9}} to {{Gerrit|ef319ea3}}
* 06:06 eileen: config revision changed from {{Gerrit|b9bc2a20}} to {{Gerrit|60ef6709}}

== 2026-03-09 ==
* 17:37 damilare: payments-wiki upgraded from {{Gerrit|5b747b97}} to {{Gerrit|f40a1153}}

== 2026-03-06 ==
* 23:39 ejegg: standalone (IPN listener) SmashPig upgraded from {{Gerrit|ed16a2ea}} to {{Gerrit|78a8e70a}}
* 21:48 ejegg: fundraising civicrm upgraded from {{Gerrit|8aadcd81}} to {{Gerrit|773d9fb9}}
* 21:13 ejegg: civicrm fundraising upgraded from {{Gerrit|a1f32ed6}} to {{Gerrit|8aadcd81}}
* 20:35 ejegg: standalone (IPN listener) SmashPig upgraded from {{Gerrit|217fc7fc}} to {{Gerrit|ed16a2ea}}
* 19:18 ejegg: civicrm upgraded from {{Gerrit|f2633c89}} to {{Gerrit|a1f32ed6}}
* 03:08 larssandergreen: civicrm upgraded from {{Gerrit|fbac3ce7}} to {{Gerrit|2bae36fa}}
* 02:09 ejegg: payments-wiki upgraded from {{Gerrit|9ae5bf60}} to {{Gerrit|5b747b97}}

== 2026-03-05 ==
* 17:53 ejegg: donorwiki upgraded from {{Gerrit|7329b41d}} to {{Gerrit|4c09db39}}
* 05:59 eileen: ivicrm upgraded from {{Gerrit|11e5a5d8}} to {{Gerrit|fbac3ce7}}
* 05:08 eileen: * civicrm upgraded from {{Gerrit|8bdce85f}} to {{Gerrit|11e5a5d8}}

== 2026-03-04 ==
* 18:52 jgleeson: tools upgraded from {{Gerrit|a3568ffc}} to {{Gerrit|e60f63b3}}
* 03:21 ejegg: payments-wiki upgraded from {{Gerrit|5e4939a3}} to {{Gerrit|9ae5bf60}}
* 03:20 ejegg: standalone (IPN listener) SmashPig upgraded from {{Gerrit|78960c68}} to {{Gerrit|217fc7fc}}

== 2026-03-03 ==
* 20:42 eileen: civicrm upgraded from {{Gerrit|b610f844}} to {{Gerrit|8bdce85f}}
* 17:09 dwisehaupt: latest php8.2 updates installed on civi1002
* 06:00 eileen: * civicrm upgraded from {{Gerrit|f4a70c82}} to {{Gerrit|b610f844}}

== 2026-03-02 ==
* 20:40 eileen: cv upgraded from {{Gerrit|dfeedcbe}} to {{Gerrit|f19e0961}}
* 18:41 ejegg: standalone (IPN listener) SmashPig upgraded from {{Gerrit|8cd1593b}} to {{Gerrit|78960c68}}
* 15:13 ejegg: donorwiki upgraded from {{Gerrit|f5d7179a}} to {{Gerrit|7329b41d}}

== 2026-02-27 ==
* 19:00 ejegg: fundraising civicrm upgraded from {{Gerrit|162bbf7c}} to {{Gerrit|f4a70c82}}
* 12:24 jgleeson: payments-wiki upgraded from {{Gerrit|f4fd71ff}} to {{Gerrit|a8a9ce78}}
* 11:25 jgleeson: payments-wiki upgraded from {{Gerrit|974af222}} to {{Gerrit|f4fd71ff}}

== 2026-02-26 ==
* 22:36 eileen: civicrm upgraded from {{Gerrit|6995dc03}} to {{Gerrit|162bbf7c}}
* 21:52 ejegg: civicrm upgraded from {{Gerrit|2a6001d3}} to {{Gerrit|6995dc03}}
* 17:44 larssandergreen: civicrm upgraded from {{Gerrit|3ac93e95}} to {{Gerrit|2a6001d3}}
* 07:14 eileen: civicrm upgraded from {{Gerrit|c3157fbf}} to {{Gerrit|3ac93e95}}

== 2026-02-25 ==
* 18:33 ejegg: fundraising civicrm upgraded from {{Gerrit|f881e026}} to {{Gerrit|c3157fbf}}
* 03:37 eileen: config revision changed from {{Gerrit|390d6434}} to {{Gerrit|a0228e6c}} turn off trustly audit

== 2026-02-24 ==
* 22:48 ejegg: payments-wiki upgraded from {{Gerrit|e5f73610}} to {{Gerrit|974af222}}
* 19:47 ejegg: payments-wiki upgraded from {{Gerrit|f5d7179a}} to {{Gerrit|e5f73610}}
* 02:06 eileen: config revision changed from {{Gerrit|71c98072}} to {{Gerrit|390d6434}} reenabled trustly audit

== 2026-02-23 ==
* 23:42 eileen: civicrm upgraded from {{Gerrit|f0710864}} to {{Gerrit|f881e026}}
* 22:28 ejegg: fundraising civicrm upgraded from {{Gerrit|9d58ce4a}} to {{Gerrit|f0710864}}
* 16:43 jgleeson: payments-wiki upgraded from {{Gerrit|0127f2d8}} to {{Gerrit|f5d7179a}}

== 2026-02-21 ==
* 01:15 dwisehaupt: updating localsettings from {{Gerrit|71c98072}} to {{Gerrit|534fbf34}} and syncing civicrm to push update for large_donation_notifications

== 2026-02-20 ==
* 20:38 ejegg: donorwiki upgraded from {{Gerrit|f7a0ee6b}} to {{Gerrit|f5d7179a}}

== 2026-02-19 ==
* 21:26 ejegg: payments-wiki upgraded from {{Gerrit|f7a0ee6b}} to {{Gerrit|0127f2d8}}
* 06:37 eileen: civicrm upgraded from {{Gerrit|ac30e19f}} to {{Gerrit|9d58ce4a}}

== 2026-02-18 ==
* 22:21 dwisehaupt: disabling apache2::mod::dump_io on civicrm role for debugging 500 errors after testing. can be re-enabled by reverting commit {{Gerrit|4b1d94399}} - [[phab:T417310|T417310]]
* 20:38 eileen: civicrm upgraded from {{Gerrit|f5020a85}} to {{Gerrit|ac30e19f}}
* 19:22 eileen: civicrm upgraded from {{Gerrit|66d2e1dd}} to {{Gerrit|f5020a85}}
* 17:48 ejegg: donorwiki upgraded from {{Gerrit|488431ec}} to {{Gerrit|f7a0ee6b}}
* 16:51 dwisehaupt: enabling apache2::mod::dump_io on civicrm role for debugging 500 errors - [[phab:T417310|T417310]]
* 06:02 eileen: civicrm upgraded from {{Gerrit|4c3cdcde}} to {{Gerrit|66d2e1dd}}
* 05:33 eileen: config revision changed from {{Gerrit|605c6946}} to {{Gerrit|368156fa}}
* 05:21 eileen: civicrm upgraded from {{Gerrit|caad5ab9}} to {{Gerrit|4c3cdcde}}

== 2026-02-17 ==
* 17:34 larssandergreen: payments-wiki upgraded from {{Gerrit|c506d590}} to {{Gerrit|488431ec}}
* 17:33 larssandergreen: donorwiki upgraded from {{Gerrit|93e0d03f}} to {{Gerrit|488431ec}}
* 03:02 eileen: civicrm upgraded from {{Gerrit|89782fc6}} to {{Gerrit|caad5ab9}}

== 2026-02-16 ==
* 20:16 eileen: civicrm upgraded from {{Gerrit|2b227403}} to {{Gerrit|89782fc6}}

== 2026-02-15 ==
* 23:55 eileen: civicrm upgraded from {{Gerrit|de8252c7}} to {{Gerrit|2b227403}}
* 21:07 eileen: civicrm upgraded from {{Gerrit|038f5bca}} to {{Gerrit|de8252c7}}

== 2026-02-13 ==
* 16:03 ejegg: payments-wiki upgraded from {{Gerrit|5793a405}} to {{Gerrit|c506d590}}
* away: SmashPig upgraded from {{Gerrit|fea03fcc}} to {{Gerrit|8cd1593b}}

== 2026-02-12 ==
* 22:49 eileen: civicrm upgraded from {{Gerrit|c6c0d453}} to {{Gerrit|038f5bca}}
* 21:45 jgleeson: payments-wiki upgraded from {{Gerrit|9dbf0ece}} to {{Gerrit|5793a405}}
* 21:15 jgleeson: payments-wiki upgraded from {{Gerrit|6c1a522f}} to {{Gerrit|9dbf0ece}}
* 19:03 larssandergreen: tools upgraded from {{Gerrit|645cf5dc}} to {{Gerrit|a3568ffc}}
* 00:34 larssandergreen: civicrm upgraded from {{Gerrit|e13111f3}} to {{Gerrit|c6c0d453}}

== 2026-02-11 ==
* 21:12 eileen: civicrm upgraded from {{Gerrit|6e57071a}} to {{Gerrit|e13111f3}}
* 06:27 eileen: civicrm upgraded from {{Gerrit|98c325dd}} to {{Gerrit|6e57071a}}
* 04:55 larssandergreen: tools upgraded from {{Gerrit|7462b8bd}} to {{Gerrit|645cf5dc}}
* 03:37 eileen: civicrm upgraded from {{Gerrit|953cf9f2}} to {{Gerrit|98c325dd}}

== 2026-02-09 ==
* 15:35 ejegg: standalone (IPN listener) SmashPig upgraded from {{Gerrit|937d6e40}} to {{Gerrit|fea03fcc}}

== 2026-02-08 ==
* 23:47 eileen: civicrm upgraded from {{Gerrit|bc3a8036}} to {{Gerrit|953cf9f2}}
* 20:46 eileen: civicrm upgraded from {{Gerrit|40189afa}} to {{Gerrit|bc3a8036}}

== 2026-02-06 ==
* 15:41 ejegg: fundraising civicrm upgraded from {{Gerrit|06842fbf}} to {{Gerrit|40189afa}}
* 06:24 eileen: civicrm upgraded from {{Gerrit|b63d7146}} to {{Gerrit|06842fbf}}

== 2026-02-05 ==
* 15:25 dwisehaupt: all eqiad hosts powered down and ready for relocation.
* 15:03 dwisehaupt: starting poweroff of eqiad hosts
* 14:48 dwisehaupt: downtimes scheduled for frack eqiad hosts and cross colo replication in prep for rack expansion - [[phab:T403035|T403035]]
* 04:30 eileen: civicrm upgraded from {{Gerrit|000dd548}} to {{Gerrit|b63d7146}}

== 2026-02-04 ==
* 23:56 eileen: civicrm upgraded from {{Gerrit|4c2870c5}} to {{Gerrit|000dd548}}
* 23:47 cstone: donorwiki upgraded from {{Gerrit|53bfb05b}} to {{Gerrit|93e0d03f}}
* 21:51 ejegg: payments-wiki upgraded from {{Gerrit|a09a4f8f}} to {{Gerrit|93e0d03f}}
* 21:46 eileen: civicrm upgraded from {{Gerrit|dd10342d}} to {{Gerrit|4c2870c5}}
* 21:22 eileen: civicrm upgraded from {{Gerrit|8aa9274a}} to {{Gerrit|dd10342d}}
* 03:48 eileen: * civicrm upgraded from {{Gerrit|14c7b7e7}} to {{Gerrit|8aa9274a}}

== 2026-02-03 ==
* 21:55 dwisehaupt: as part of dns cleanup, we have removed the old civi1002.wikimedia.org entry. folks should be using civicrm.wm.o but there is a chance of super old bookmarks still being around. we can reinstate if it becomes an issue.
* 21:05 eileen: config revision changed from {{Gerrit|23b2d9b6}} to {{Gerrit|45d40cf1}}
* 21:01 larssandergreen: civicrm upgraded from {{Gerrit|10ab3659}} to {{Gerrit|14c7b7e7}}
* 05:49 eileen: config revision changed from {{Gerrit|f348441d}} to {{Gerrit|23b2d9b6}}
* 05:48 eileen: civicrm upgraded from {{Gerrit|a097bb3d}} to {{Gerrit|10ab3659}}
* 01:16 wfan: civicrm upgraded from {{Gerrit|5b9f3cd4}} to {{Gerrit|a097bb3d}}
* 00:24 wfan: donorwiki upgraded from {{Gerrit|3ffc70f0}} to {{Gerrit|53bfb05b}}

== 2026-02-02 ==
* 23:50 wfan: payments-wiki upgraded from {{Gerrit|c035aa84}} to {{Gerrit|53bfb05b}}
* 20:22 cstone: civicrm upgraded from {{Gerrit|f91f955b}} to {{Gerrit|5b9f3cd4}}
* 18:55 ejegg: standalone (IPN listener) SmashPig upgraded from {{Gerrit|b42f2de4}} to {{Gerrit|937d6e40}}
* 17:01 wfan: payments-wiki upgraded from {{Gerrit|c5cadd72}} to {{Gerrit|c035aa84}}
* 15:26 ejegg: fundraising civicrm upgraded from {{Gerrit|611d18de}} to {{Gerrit|f91f955b}}

== 2026-01-30 ==
* 17:02 larssandergreen: civicrm upgraded from {{Gerrit|79e4424e}} to {{Gerrit|611d18de}}
* 04:18 cstone: civicrm upgraded from {{Gerrit|fe1af57a}} to {{Gerrit|79e4424e}}
* 02:51 eileen: civicrm upgraded from {{Gerrit|ff772bee}} to {{Gerrit|fe1af57a}}
* 02:28 eileen: civicrm upgraded from {{Gerrit|bcf976ae}} to {{Gerrit|ff772bee}}

== 2026-01-29 ==
* 22:16 eileen: civicrm upgraded from {{Gerrit|5d121c63}} to {{Gerrit|bcf976ae}}
* 20:17 eileen: civicrm upgraded from {{Gerrit|ebcfd009}} to {{Gerrit|5d121c63}}
* 20:09 eileen: civicrm upgraded from {{Gerrit|5c065c4e}} to {{Gerrit|ebcfd009}}
* 18:08 wfan: payments-wiki upgraded from {{Gerrit|81d9f614}} to {{Gerrit|c5cadd72}}

== 2026-01-28 ==
* 19:36 jgleeson: SmashPig upgraded from {{Gerrit|96a6224d}} to {{Gerrit|b42f2de4}}
* 18:53 jgleeson: civicrm upgraded from {{Gerrit|56c222da}} to {{Gerrit|5c065c4e}}
* 18:53 jgleeson: SmashPig upgraded from {{Gerrit|96a6224d}} to {{Gerrit|b42f2de4}}
* 14:41 jgleeson: payments-wiki upgraded from {{Gerrit|24915bdb}} to {{Gerrit|81d9f614}}
* 07:30 eileen: civicrm upgraded from {{Gerrit|600b21a6}} to {{Gerrit|56c222da}}
* 06:51 eileen: * civicrm upgraded from {{Gerrit|32f9a10d}} to {{Gerrit|600b21a6}}
* 04:35 eileen: config revision changed from {{Gerrit|ed0808a9}} to {{Gerrit|ef6ef5f2}}
* 03:42 eileen: civicrm upgraded from {{Gerrit|64267a34}} to {{Gerrit|32f9a10d}}
* 02:13 eileen: civicrm upgraded from {{Gerrit|7299615a}} to {{Gerrit|64267a34}}

== 2026-01-27 ==
* 21:12 larssandergreen: tools upgraded from {{Gerrit|84323460}} to {{Gerrit|7462b8bd}}
* 05:57 cstone: civicrm upgraded from {{Gerrit|19f94835}} to {{Gerrit|75f443b5}}

== 2026-01-26 ==
* 17:15 damilare: smashpig upgraded from {{Gerrit|8b4ebf34}} to {{Gerrit|96a6224d}}
* 16:32 larssandergreen: tools upgraded from {{Gerrit|c75f7625}} to {{Gerrit|84323460}}
* 01:43 eileen: config revision changed from {{Gerrit|2f71107f}} to {{Gerrit|ed0808a9}} switch to php dlocal downloader (now weekend is mostly over) -

== 2026-01-25 ==
* 21:47 eileen: config revision changed from {{Gerrit|23023984}} to {{Gerrit|2f71107f}}

== 2026-01-24 ==
* 02:19 cstone: civicrm upgraded from {{Gerrit|f7064a46}} to {{Gerrit|19f94835}}
* 00:55 bd808: Testing #wikimedia-fundraising SAL integration ([[phab:T415389|T415389]])


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>mkrblwl047g7lf0ypqas0ijk8yr9adp</sha1>
    </revision>
  </page>
  <page>
    <title>Tool:Etherpad-backup/Log</title>
    <ns>116</ns>
    <id>460070</id>
    <revision>
      <id>2408780</id>
      <parentid>2408767</parentid>
      <timestamp>2026-05-03T12:00:42Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>BugTriage-Collection archived by BryanDavis</comment>
      <origin>2408780</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="17200" sha1="1sbj11dgaru3eji6oqlxgv12dwhtwkj" xml:space="preserve">=== 2026-05-03 ===
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>1sbj11dgaru3eji6oqlxgv12dwhtwkj</sha1>
    </revision>
    <revision>
      <id>2408781</id>
      <parentid>2408780</parentid>
      <timestamp>2026-05-03T12:00:44Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>DrTrigonbotfalsepositivesfilter archived by BryanDavis</comment>
      <origin>2408781</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="17356" sha1="0y3qiqemv0fcddd0inp1whdghalv4rj" xml:space="preserve">=== 2026-05-03 ===
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>0y3qiqemv0fcddd0inp1whdghalv4rj</sha1>
    </revision>
    <revision>
      <id>2408782</id>
      <parentid>2408781</parentid>
      <timestamp>2026-05-03T12:00:47Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>GD0bnahDcM archived by BryanDavis</comment>
      <origin>2408782</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="17470" sha1="p66xwk37yybo0fv3w4s7tniayrdhiug" xml:space="preserve">=== 2026-05-03 ===
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>p66xwk37yybo0fv3w4s7tniayrdhiug</sha1>
    </revision>
    <revision>
      <id>2408783</id>
      <parentid>2408782</parentid>
      <timestamp>2026-05-03T12:00:48Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>GLAM-Wiki-CH_OpenRefine_PAWS archived by BryanDavis</comment>
      <origin>2408783</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="17620" sha1="0j3d7ynere7c78e4o55ml3pph5dqlcy" xml:space="preserve">=== 2026-05-03 ===
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>0j3d7ynere7c78e4o55ml3pph5dqlcy</sha1>
    </revision>
    <revision>
      <id>2408784</id>
      <parentid>2408783</parentid>
      <timestamp>2026-05-03T12:00:50Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>GLAMcampAmsterdamFri archived by BryanDavis</comment>
      <origin>2408784</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="17754" sha1="du0wlihum8w203twe3ipb54mqnupu8z" xml:space="preserve">=== 2026-05-03 ===
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>du0wlihum8w203twe3ipb54mqnupu8z</sha1>
    </revision>
    <revision>
      <id>2408785</id>
      <parentid>2408784</parentid>
      <timestamp>2026-05-03T12:00:52Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>IRCBot-Messages archived by BryanDavis</comment>
      <origin>2408785</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="17878" sha1="kv1d2wuutktu8gehlyv9ciq7co0x7nd" xml:space="preserve">=== 2026-05-03 ===
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>kv1d2wuutktu8gehlyv9ciq7co0x7nd</sha1>
    </revision>
    <revision>
      <id>2408786</id>
      <parentid>2408785</parentid>
      <timestamp>2026-05-03T12:00:54Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>WBUG_2023_07_27 archived by BryanDavis</comment>
      <origin>2408786</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="18002" sha1="tljjjavxi1dtwy6gsvlm5cdjzkthuj3" xml:space="preserve">=== 2026-05-03 ===
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>tljjjavxi1dtwy6gsvlm5cdjzkthuj3</sha1>
    </revision>
    <revision>
      <id>2408787</id>
      <parentid>2408786</parentid>
      <timestamp>2026-05-03T12:00:56Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>WBUG_2023_09_28 archived by BryanDavis</comment>
      <origin>2408787</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="18126" sha1="rqrlk21w9ax34ov5dc5dre70h5liopf" xml:space="preserve">=== 2026-05-03 ===
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>rqrlk21w9ax34ov5dc5dre70h5liopf</sha1>
    </revision>
    <revision>
      <id>2408788</id>
      <parentid>2408787</parentid>
      <timestamp>2026-05-03T12:02:32Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>DrTrigonbotfalsepositivesfilter archived by BryanDavis</comment>
      <origin>2408788</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="18282" sha1="2vp3eh05qt8uhgkxk1e1v2a9idoua63" xml:space="preserve">=== 2026-05-03 ===
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>2vp3eh05qt8uhgkxk1e1v2a9idoua63</sha1>
    </revision>
    <revision>
      <id>2408789</id>
      <parentid>2408788</parentid>
      <timestamp>2026-05-03T12:02:34Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>GD0bnahDcM archived by BryanDavis</comment>
      <origin>2408789</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="18396" sha1="t39a9mdrcapm566tu5y7yb1kgiis456" xml:space="preserve">=== 2026-05-03 ===
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>t39a9mdrcapm566tu5y7yb1kgiis456</sha1>
    </revision>
    <revision>
      <id>2408790</id>
      <parentid>2408789</parentid>
      <timestamp>2026-05-03T12:02:35Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>GLAM-Wiki-CH_OpenRefine_PAWS archived by BryanDavis</comment>
      <origin>2408790</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="18546" sha1="4cfgu7lgc2ryye0ps7al1w6ruuzxld8" xml:space="preserve">=== 2026-05-03 ===
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>4cfgu7lgc2ryye0ps7al1w6ruuzxld8</sha1>
    </revision>
    <revision>
      <id>2408791</id>
      <parentid>2408790</parentid>
      <timestamp>2026-05-03T12:02:37Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>GLAMcampAmsterdamFri archived by BryanDavis</comment>
      <origin>2408791</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="18680" sha1="6v8s5pbc0tkv815owmmqrzwa9828jyf" xml:space="preserve">=== 2026-05-03 ===
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>6v8s5pbc0tkv815owmmqrzwa9828jyf</sha1>
    </revision>
    <revision>
      <id>2408792</id>
      <parentid>2408791</parentid>
      <timestamp>2026-05-03T12:02:39Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>IRCBot-Messages archived by BryanDavis</comment>
      <origin>2408792</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="18804" sha1="pz3gtfedlz5zud8r6q13gidd4nndre5" xml:space="preserve">=== 2026-05-03 ===
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>pz3gtfedlz5zud8r6q13gidd4nndre5</sha1>
    </revision>
    <revision>
      <id>2408793</id>
      <parentid>2408792</parentid>
      <timestamp>2026-05-03T12:02:42Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>WBUG_2023_07_27 archived by BryanDavis</comment>
      <origin>2408793</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="18928" sha1="0ks20ighconin5px4u7en1titfit2je" xml:space="preserve">=== 2026-05-03 ===
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>0ks20ighconin5px4u7en1titfit2je</sha1>
    </revision>
    <revision>
      <id>2408794</id>
      <parentid>2408793</parentid>
      <timestamp>2026-05-03T12:02:44Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>WBUG_2023_09_28 archived by BryanDavis</comment>
      <origin>2408794</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="19052" sha1="h3mvwhavifpth2f8x8se9ij7c2kk15f" xml:space="preserve">=== 2026-05-03 ===
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>h3mvwhavifpth2f8x8se9ij7c2kk15f</sha1>
    </revision>
    <revision>
      <id>2408795</id>
      <parentid>2408794</parentid>
      <timestamp>2026-05-03T12:02:45Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>VertalingWikiIt archived by BryanDavis</comment>
      <origin>2408795</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="19176" sha1="kqerlrlt9jef25uitfxys0ktqoqzzkr" xml:space="preserve">=== 2026-05-03 ===
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>kqerlrlt9jef25uitfxys0ktqoqzzkr</sha1>
    </revision>
    <revision>
      <id>2408796</id>
      <parentid>2408795</parentid>
      <timestamp>2026-05-03T12:02:47Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>WIKISOO3 archived by BryanDavis</comment>
      <origin>2408796</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="19286" sha1="ddl1sthsl4lgpewjspk4o3ll8ydjeaq" xml:space="preserve">=== 2026-05-03 ===
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>ddl1sthsl4lgpewjspk4o3ll8ydjeaq</sha1>
    </revision>
    <revision>
      <id>2408798</id>
      <parentid>2408796</parentid>
      <timestamp>2026-05-03T13:11:48Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>0WdKUDRA58 archived by BryanDavis</comment>
      <origin>2408798</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="19400" sha1="tdqlre36ig498m24661u8sur9esgoau" xml:space="preserve">=== 2026-05-03 ===
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>tdqlre36ig498m24661u8sur9esgoau</sha1>
    </revision>
    <revision>
      <id>2408799</id>
      <parentid>2408798</parentid>
      <timestamp>2026-05-03T13:11:49Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>1.25wmf15-perf-regression archived by BryanDavis</comment>
      <origin>2408799</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="19544" sha1="b7ajrja6am7wg3n70q9wtub5uvjgh98" xml:space="preserve">=== 2026-05-03 ===
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>b7ajrja6am7wg3n70q9wtub5uvjgh98</sha1>
    </revision>
    <revision>
      <id>2408800</id>
      <parentid>2408799</parentid>
      <timestamp>2026-05-03T13:11:49Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>1.27.0-wmf.13-issues archived by BryanDavis</comment>
      <origin>2408800</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="19678" sha1="shmafg4923v7a5fml32pmt3879g56fl" xml:space="preserve">=== 2026-05-03 ===
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>shmafg4923v7a5fml32pmt3879g56fl</sha1>
    </revision>
    <revision>
      <id>2408801</id>
      <parentid>2408800</parentid>
      <timestamp>2026-05-03T13:11:50Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>1.36-release archived by BryanDavis</comment>
      <origin>2408801</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="19796" sha1="huztfb0elblz7r155sqhpmaeh7t2l28" xml:space="preserve">=== 2026-05-03 ===
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>huztfb0elblz7r155sqhpmaeh7t2l28</sha1>
    </revision>
    <revision>
      <id>2408802</id>
      <parentid>2408801</parentid>
      <timestamp>2026-05-03T13:11:51Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>119-beta archived by BryanDavis</comment>
      <origin>2408802</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="19906" sha1="bbt2x6wcq7oyppc0y1waocig2rxuwdo" xml:space="preserve">=== 2026-05-03 ===
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>bbt2x6wcq7oyppc0y1waocig2rxuwdo</sha1>
    </revision>
    <revision>
      <id>2408803</id>
      <parentid>2408802</parentid>
      <timestamp>2026-05-03T13:11:52Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>119917 archived by BryanDavis</comment>
      <origin>2408803</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="20012" sha1="n1k199fwmz9td0nbm3z15a28kba7vx2" xml:space="preserve">=== 2026-05-03 ===
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>n1k199fwmz9td0nbm3z15a28kba7vx2</sha1>
    </revision>
    <revision>
      <id>2408804</id>
      <parentid>2408803</parentid>
      <timestamp>2026-05-03T13:11:53Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>119deployment archived by BryanDavis</comment>
      <origin>2408804</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="20132" sha1="d1h1i43j9zfiekxrawp1rw5y35fanij" xml:space="preserve">=== 2026-05-03 ===
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>d1h1i43j9zfiekxrawp1rw5y35fanij</sha1>
    </revision>
    <revision>
      <id>2408805</id>
      <parentid>2408804</parentid>
      <timestamp>2026-05-03T13:11:54Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>119triage archived by BryanDavis</comment>
      <origin>2408805</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="20244" sha1="d3v9pr6z46au1iqoat6a9pyc6ngsqsf" xml:space="preserve">=== 2026-05-03 ===
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>d3v9pr6z46au1iqoat6a9pyc6ngsqsf</sha1>
    </revision>
    <revision>
      <id>2408806</id>
      <parentid>2408805</parentid>
      <timestamp>2026-05-03T13:11:55Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>120wmf2deployment archived by BryanDavis</comment>
      <origin>2408806</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="20372" sha1="hqmmwyjb398n3b980s9m3bg94vhac5m" xml:space="preserve">=== 2026-05-03 ===
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>hqmmwyjb398n3b980s9m3bg94vhac5m</sha1>
    </revision>
    <revision>
      <id>2408807</id>
      <parentid>2408806</parentid>
      <timestamp>2026-05-03T13:11:55Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>123 archived by BryanDavis</comment>
      <origin>2408807</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="20472" sha1="o9f7d0cjdbwkatelk1ltwodtpsbjxfc" xml:space="preserve">=== 2026-05-03 ===
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>o9f7d0cjdbwkatelk1ltwodtpsbjxfc</sha1>
    </revision>
    <revision>
      <id>2408808</id>
      <parentid>2408807</parentid>
      <timestamp>2026-05-03T13:14:33Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>0WdKUDRA58 archived by BryanDavis</comment>
      <origin>2408808</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="20586" sha1="pprx6friq3si884lguwtponmcjugi6m" xml:space="preserve">=== 2026-05-03 ===
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>pprx6friq3si884lguwtponmcjugi6m</sha1>
    </revision>
    <revision>
      <id>2408809</id>
      <parentid>2408808</parentid>
      <timestamp>2026-05-03T13:14:34Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>1.25wmf15-perf-regression archived by BryanDavis</comment>
      <origin>2408809</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="20730" sha1="g6u6j1e75xwynvrqp1g1bko9b7hibv5" xml:space="preserve">=== 2026-05-03 ===
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>g6u6j1e75xwynvrqp1g1bko9b7hibv5</sha1>
    </revision>
    <revision>
      <id>2408810</id>
      <parentid>2408809</parentid>
      <timestamp>2026-05-03T13:14:34Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>1.27.0-wmf.13-issues archived by BryanDavis</comment>
      <origin>2408810</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="20864" sha1="innvwmwmm8amk186v07twmkefcfjuk6" xml:space="preserve">=== 2026-05-03 ===
* 13:14 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>innvwmwmm8amk186v07twmkefcfjuk6</sha1>
    </revision>
    <revision>
      <id>2408811</id>
      <parentid>2408810</parentid>
      <timestamp>2026-05-03T13:14:35Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>1.36-release archived by BryanDavis</comment>
      <origin>2408811</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="20982" sha1="k0w3tigdhf7ti4m8qjjyf9er0s7wnse" xml:space="preserve">=== 2026-05-03 ===
* 13:14 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>k0w3tigdhf7ti4m8qjjyf9er0s7wnse</sha1>
    </revision>
    <revision>
      <id>2408812</id>
      <parentid>2408811</parentid>
      <timestamp>2026-05-03T13:14:36Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>119-beta archived by BryanDavis</comment>
      <origin>2408812</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="21092" sha1="s7b4jqbc3hi8kavt3ih8jlemm0qj15w" xml:space="preserve">=== 2026-05-03 ===
* 13:14 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>s7b4jqbc3hi8kavt3ih8jlemm0qj15w</sha1>
    </revision>
    <revision>
      <id>2408813</id>
      <parentid>2408812</parentid>
      <timestamp>2026-05-03T13:14:37Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>119917 archived by BryanDavis</comment>
      <origin>2408813</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="21198" sha1="8or6uy3uxdokhcq1ib41gif0oqmkq2b" xml:space="preserve">=== 2026-05-03 ===
* 13:14 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>8or6uy3uxdokhcq1ib41gif0oqmkq2b</sha1>
    </revision>
    <revision>
      <id>2408814</id>
      <parentid>2408813</parentid>
      <timestamp>2026-05-03T13:14:38Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>119deployment archived by BryanDavis</comment>
      <origin>2408814</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="21318" sha1="5i7pbg9fqgxe5n2ta5wp3op2x6ku48n" xml:space="preserve">=== 2026-05-03 ===
* 13:14 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>5i7pbg9fqgxe5n2ta5wp3op2x6ku48n</sha1>
    </revision>
    <revision>
      <id>2408815</id>
      <parentid>2408814</parentid>
      <timestamp>2026-05-03T13:14:38Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>119triage archived by BryanDavis</comment>
      <origin>2408815</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="21430" sha1="fl1kva0ap97yf0fny2bnspi59y4oj8x" xml:space="preserve">=== 2026-05-03 ===
* 13:14 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>fl1kva0ap97yf0fny2bnspi59y4oj8x</sha1>
    </revision>
    <revision>
      <id>2408816</id>
      <parentid>2408815</parentid>
      <timestamp>2026-05-03T13:14:39Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>120wmf2deployment archived by BryanDavis</comment>
      <origin>2408816</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="21558" sha1="3pd2i5ue020dzthkruw4ma5hxrfhc0h" xml:space="preserve">=== 2026-05-03 ===
* 13:14 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>3pd2i5ue020dzthkruw4ma5hxrfhc0h</sha1>
    </revision>
    <revision>
      <id>2408817</id>
      <parentid>2408816</parentid>
      <timestamp>2026-05-03T13:14:40Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>123 archived by BryanDavis</comment>
      <origin>2408817</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="21658" sha1="4ci8i7btr4je4vy91w8h7sre1ytyhky" xml:space="preserve">=== 2026-05-03 ===
* 13:14 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>4ci8i7btr4je4vy91w8h7sre1ytyhky</sha1>
    </revision>
    <revision>
      <id>2408820</id>
      <parentid>2408817</parentid>
      <timestamp>2026-05-03T13:56:14Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>bd808-etherpad-backup-test archived by BryanDavis</comment>
      <origin>2408820</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="21804" sha1="npkh9hd7oewl7wyie66vv5cmk8od20e" xml:space="preserve">=== 2026-05-03 ===
* 13:56 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>npkh9hd7oewl7wyie66vv5cmk8od20e</sha1>
    </revision>
    <revision>
      <id>2408821</id>
      <parentid>2408820</parentid>
      <timestamp>2026-05-03T13:56:16Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>bd808_test/path/?query=param archived by BryanDavis</comment>
      <origin>2408821</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="21954" sha1="skp5c1h4moiatgdri7cpvm1qnn7nx2b" xml:space="preserve">=== 2026-05-03 ===
* 13:56 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 13:56 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>skp5c1h4moiatgdri7cpvm1qnn7nx2b</sha1>
    </revision>
    <revision>
      <id>2408822</id>
      <parentid>2408821</parentid>
      <timestamp>2026-05-03T13:56:19Z</timestamp>
      <contributor>
        <username>EtherpadBackupBot</username>
        <id>54504</id>
      </contributor>
      <comment>bd808_test/path/?query=param archived by BryanDavis</comment>
      <origin>2408822</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="22104" sha1="eo54hyt012rfvgf6jusvtngc4512cwb" xml:space="preserve">=== 2026-05-03 ===
* 13:56 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 13:56 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 13:56 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:14 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/123]] was archived as [[etherpadbackup:123]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/120wmf2deployment]] was archived as [[etherpadbackup:120wmf2deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119triage]] was archived as [[etherpadbackup:119triage]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119deployment]] was archived as [[etherpadbackup:119deployment]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119917]] was archived as [[etherpadbackup:119917]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/119-beta]] was archived as [[etherpadbackup:119-beta]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.36-release]] was archived as [[etherpadbackup:1.36-release]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.27.0-wmf.13-issues]] was archived as [[etherpadbackup:1.27.0-wmf.13-issues]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/1.25wmf15-perf-regression]] was archived as [[etherpadbackup:1.25wmf15-perf-regression]] by [[User:BryanDavis|BryanDavis]]
* 13:11 [[etherpad:p/0WdKUDRA58]] was archived as [[etherpadbackup:0WdKUDRA58]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WIKISOO3]] was archived as [[etherpadbackup:WIKISOO3]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/VertalingWikiIt]] was archived as [[etherpadbackup:VertalingWikiIt]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:02 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_09_28]] was archived as [[etherpadbackup:WBUG_2023_09_28]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/WBUG_2023_07_27]] was archived as [[etherpadbackup:WBUG_2023_07_27]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/IRCBot-Messages]] was archived as [[etherpadbackup:IRCBot-Messages]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAMcampAmsterdamFri]] was archived as [[etherpadbackup:GLAMcampAmsterdamFri]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GLAM-Wiki-CH_OpenRefine_PAWS]] was archived as [[etherpadbackup:GLAM-Wiki-CH_OpenRefine_PAWS]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/GD0bnahDcM]] was archived as [[etherpadbackup:GD0bnahDcM]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/DrTrigonbotfalsepositivesfilter]] was archived as [[etherpadbackup:DrTrigonbotfalsepositivesfilter]] by [[User:BryanDavis|BryanDavis]]
* 12:00 [[etherpad:p/BugTriage-Collection]] was archived as [[etherpadbackup:BugTriage-Collection]] by [[User:BryanDavis|BryanDavis]]
* 09:25 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 09:18 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:46 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:28 [[etherpad:p/prodlikevagrant-hackathon-planning]] was archived as [[etherpadbackup:prodlikevagrant-hackathon-planning]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-02 ===
* 13:01 [[etherpad:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] was archived as [[etherpadbackup:p/wpcon16_Wikipedianer_%26_%E2%80%9EMateriale_Textkultu]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 07:18 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 06:58 [[etherpad:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] was archived as [[etherpadbackup:p/test-*%2C%3F%2C\%2C%2F%2C%25%2C&gt;%2C&lt;%2C🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-05-01 ===
* 18:12 [[etherpad:p/10myths]] was archived as [[etherpadbackup:p/10myths]] by [[User:Krinkle|Krinkle]]
* 15:30 [[etherpad:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] was archived as [[etherpadbackup:p/test-*,?,\,/,%,&gt;,&lt;,🤖💬]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 14:50 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:Jon Harald Søby|Jon Harald Søby]]
* 14:42 [[etherpad:p/PoGo_Friendship_Codes]] was archived as [[etherpadbackup:p/PoGo_Friendship_Codes]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:51 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:16 [[etherpad:p/Wikimedia_Hackathon_2026_Opening]] was archived as [[etherpadbackup:p/Wikimedia_Hackathon_2026_Opening]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 13:07 [[etherpad:p/copyvios-todo]] was archived as [[etherpadbackup:p/copyvios-todo]] by [[User:Chlod|Chlod]]
* 10:05 [[etherpad:p/hhvmdeploy]] was archived as [[etherpadbackup:p/hhvmdeploy]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-29 ===
* 00:49 [[etherpad:p/better-security-patch-management]] was archived as [[etherpadbackup:p/better-security-patch-management]] by [[User:BDavis (WMF)|BDavis (WMF)]]
* 00:49 [[etherpad:p/rooks-questions-to-alex]] was archived as [[etherpadbackup:p/rooks-questions-to-alex]] by [[User:BDavis (WMF)|BDavis (WMF)]]

=== 2026-04-27 ===
* 04:44 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 03:46 [[etherpad:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] was archived as [[etherpadbackup:p/wmh2023-Overview_of_technical_areas_&amp;_projects]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_18/04]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_18/04]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/open_meeting_COVID-19_04/05]] was archived as [[etherpadbackup:p/open_meeting_COVID-19_04/05]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/aswikimeet/july-2024]] was archived as [[etherpadbackup:p/aswikimeet/july-2024]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikisource_Meetup_29/09/2020]] was archived as [[etherpadbackup:p/Wikisource_Meetup_29/09/2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] was archived as [[etherpadbackup:p/Wikimedians_of_Nepal/Events/Meetup_February_2020]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/Wikimedia_LGBT+/Governance/2023-10-13]] was archived as [[etherpadbackup:p/Wikimedia_LGBT+/Governance/2023-10-13]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopD]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopD]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopC]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopC]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopB]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopB]] by [[User:BryanDavis|BryanDavis]]
* 03:45 [[etherpad:p/WikiCendekia2026/WarkopA]] was archived as [[etherpadbackup:p/WikiCendekia2026/WarkopA]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Stolpersteine_nach_WikiData_(01/2025]] was archived as [[etherpadbackup:p/Stolpersteine_nach_WikiData_(01/2025]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Sommer_24/8,_1._August]] was archived as [[etherpadbackup:p/Sommer_24/8,_1._August]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] was archived as [[etherpadbackup:p/SVG_Translation_Campaign_2019_India/Event/Patiala]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Meetup/Cape_Town/Cape_Town_22]] was archived as [[etherpadbackup:p/Meetup/Cape_Town/Cape_Town_22]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] was archived as [[etherpadbackup:p/Mach_mit!/2._Termin_Mentorinnen-Netzwerk]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] was archived as [[etherpadbackup:p/Kopdar_Wikisource_(Wikipabukon)_06/08/23]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] was archived as [[etherpadbackup:p/Igbo_Wikimedians_Community/Movement_Charter_Commun]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_C]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_C]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_B]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_B]] by [[User:BryanDavis|BryanDavis]]
* 03:42 [[etherpad:p/GLAM-Treffen_2020/Gruppe_A]] was archived as [[etherpadbackup:p/GLAM-Treffen_2020/Gruppe_A]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] was archived as [[etherpadbackup:p/CEE_Talks/Update_CEE_Hub_/_Outlook_CEE_Meeting_202]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] was archived as [[etherpadbackup:p/CEE_Talks/Outlook_CEE_Meeting_2022_V2]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Wikispore]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Wikispore]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Translatable_modules]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Translatable_modules]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Opportunities_for_1Lib1Ref]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Movement_Strategy]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Movement_Strategy]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Lua_templates_for_beginners]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Lua_templates_for_beginners]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/Help_your_community_grow]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/Help_your_community_grow]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/CEE_Meeting_2020/101_Ways_to_Contribute]] was archived as [[etherpadbackup:p/CEE_Meeting_2020/101_Ways_to_Contribute]] by [[User:BryanDavis|BryanDavis]]
* 03:40 [[etherpad:p/AsWikiMentoringProject2024/Oct-Dec]] was archived as [[etherpadbackup:p/AsWikiMentoringProject2024/Oct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:16 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:15 [[etherpad:p/AsWikiMentoringProject2024%2FOct-Dec]] was deleted as [[etherpadbackup:p/AsWikiMentoringProject2024%2FOct-Dec]] by [[User:BryanDavis|BryanDavis]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:wmh2023-Overview_of_technical_areas_%26_projects|wmh2023-Overview_of_technical_areas_%26_projects]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_20%2F04|open_meeting_COVID-19_20%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_18%2F04|open_meeting_COVID-19_18%2F04]]
* 03:12 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:open_meeting_COVID-19_04%2F05|open_meeting_COVID-19_04%2F05]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:aswikimeet%2Fjuly-2024|aswikimeet%2Fjuly-2024]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikisource_Meetup_29%2F09%2F2020|Wikisource_Meetup_29%2F09%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020|Wikimedians_of_Nepal%2FEvents%2FMeetup_February_2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Wikimedia_LGBT+%2FGovernance%2F2023-10-13|Wikimedia_LGBT+%2FGovernance%2F2023-10-13]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopD|WikiCendekia2026%2FWarkopD]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopC|WikiCendekia2026%2FWarkopC]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopB|WikiCendekia2026%2FWarkopB]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:WikiCendekia2026%2FWarkopA|WikiCendekia2026%2FWarkopA]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Summary%2F_Hypothesis|Summary%2F_Hypothesis]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(12%2F2024)|Stolpersteine_nach_WikiData_(12%2F2024)]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Stolpersteine_nach_WikiData_(01%2F2025|Stolpersteine_nach_WikiData_(01%2F2025]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Sommer_24%2F8,_1._August|Sommer_24%2F8,_1._August]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala|SVG_Translation_Campaign_2019_India%2FEvent%2FPatiala]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Meetup%2FCape_Town%2FCape_Town_22|Meetup%2FCape_Town%2FCape_Town_22]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk|Mach_mit!%2F2._Termin_Mentorinnen-Netzwerk]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23|Kopdar_Wikisource_(Wikipabukon)_06%2F08%2F23]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Igbo_Wikimedians_Community%2FMovement_Charter_Commun|Igbo_Wikimedians_Community%2FMovement_Charter_Commun]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_C|GLAM-Treffen_2020%2FGruppe_C]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_B|GLAM-Treffen_2020%2FGruppe_B]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:GLAM-Treffen_2020%2FGruppe_A|GLAM-Treffen_2020%2FGruppe_A]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:Commons_Photographers_30%2F05%2F2020|Commons_Photographers_30%2F05%2F2020]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:ChristianSW%2FNotizen|ChristianSW%2FNotizen]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202|CEE_Talks%2FUpdate_CEE_Hub_%2F_Outlook_CEE_Meeting_202]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Talks%2FOutlook_CEE_Meeting_2022_V2|CEE_Talks%2FOutlook_CEE_Meeting_2022_V2]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FWikispore|CEE_Meeting_2020%2FWikispore]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FTranslatable_modules|CEE_Meeting_2020%2FTranslatable_modules]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref|CEE_Meeting_2020%2FOpportunities_for_1Lib1Ref]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FMovement_Strategy|CEE_Meeting_2020%2FMovement_Strategy]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FLua_templates_for_beginners|CEE_Meeting_2020%2FLua_templates_for_beginners]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2FHelp_your_community_grow|CEE_Meeting_2020%2FHelp_your_community_grow]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:CEE_Meeting_2020%2F101_Ways_to_Contribute|CEE_Meeting_2020%2F101_Ways_to_Contribute]]
* 03:11 [[etherpad:p/BryanDavis]] was deleted as [[etherpadbackup:p/BryanDavis]] by [[User:AsWikiMentoringProject2024%2FOct-Dec|AsWikiMentoringProject2024%2FOct-Dec]]
* 01:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:51 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:50 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:49 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:44 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:43 [[etherpad:p/foo]] was archived as [[etherpadbackup:p/foo]] by [[User:EtherpadBackupBot|EtherpadBackupBot]]
* 00:05 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-26 ===
* 23:25 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 21:58 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:51 [[etherpad:p/bd808_test/path/?query=param]] was archived as [[etherpadbackup:p/bd808_test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808 test/path/?query=param]] was archived as [[etherpadbackup:p/bd808 test/path/?query=param]] by [[User:BryanDavis|BryanDavis]]
* 17:22 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 16:17 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]

=== 2026-04-20 ===
* 02:42 [[etherpad:p/bd808-etherpad-backup-test]] was archived as [[etherpadbackup:p/bd808-etherpad-backup-test]] by [[User:BryanDavis|BryanDavis]]
* 02:36 [[etherpad:p/bd808-etherpad-backup-test]] was archived by [[User:BryanDavis]]</text>
      <sha1>eo54hyt012rfvgf6jusvtngc4512cwb</sha1>
    </revision>
  </page>
  <page>
    <title>Nova Resource:Tools.etherpad-backup/SAL</title>
    <ns>498</ns>
    <id>460123</id>
    <revision>
      <id>2408823</id>
      <parentid>2408766</parentid>
      <timestamp>2026-05-03T13:59:58Z</timestamp>
      <contributor>
        <username>Stashbot</username>
        <id>7414</id>
      </contributor>
      <comment>wmbot~bd808@tools-bastion-14: Built image from 746ac0bd and deployed (T417207)</comment>
      <origin>2408823</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="1041" sha1="4yfqc7oemrbzz34b1vk14b6farxvphn" xml:space="preserve">=== 2026-05-03 ===
* 13:59 wmbot~bd808@tools-bastion-14: Built image from {{Gerrit|746ac0bd}} and deployed ([[phab:T417207|T417207]])
* 09:25 wmbot~bd808@tools-bastion-14: Deployed new image ([[phab:T423905|T423905]], [[phab:T425201|T425201]])
* 09:23 wmbot~bd808@tools-bastion-14: Built new image from git hash {{Gerrit|0c4bdc2f}} ([[phab:T423905|T423905]], [[phab:T425201|T425201]])
* 07:11 wmbot~bd808@tools-bastion-14: Built image from {{Gerrit|c94b08ea}} and deployed ([[phab:T425224|T425224]])

=== 2026-05-02 ===
* 07:17 wmbot~bd808@tools-bastion-14: Built image from {{Gerrit|ae85975c}} and deployed ([[phab:T423905|T423905]])

=== 2026-05-01 ===
* 15:05 wmbot~bd808@tools-bastion-14: Built image from {{Gerrit|db847baa}} and deployed ([[phab:T425111|T425111]])
* 13:03 wmbot~bd808@tools-bastion-14: Built image from {{Gerrit|bab09762}} and deployed ([[phab:T422958|T422958]])
* 10:17 wmbot~bd808@tools-bastion-14: Built and deployed image from {{Gerrit|ecf5892c}} ([[phab:T422958|T422958]])


&lt;noinclude&gt;[[Category:SAL]]&lt;/noinclude&gt;</text>
      <sha1>4yfqc7oemrbzz34b1vk14b6farxvphn</sha1>
    </revision>
  </page>
</mediawiki>